2024-02-20T12:36:08.922 INFO:root:teuthology version: 0.0.1.dev260+gb1dac55 2024-02-20T12:36:08.922 DEBUG:teuthology.run:Teuthology command: teuthology --archive /home/teuthworker/archive/yuriw-2024-02-19_20:21:18-fs-wip-yuri5-testing-2024-02-16-1120-quincy-distro-default-smithi/7567673 --description fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/pacific 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-inline/no 4-verify} 2-client 3-upgrade-with-workload 4-verify}} --name yuriw-2024-02-19_20:21:18-fs-wip-yuri5-testing-2024-02-16-1120-quincy-distro-default-smithi --verbose --owner scheduled_yuriw@teuthology -- /home/teuthworker/archive/yuriw-2024-02-19_20:21:18-fs-wip-yuri5-testing-2024-02-16-1120-quincy-distro-default-smithi/7567673/orig.config.yaml 2024-02-20T12:36:08.972 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-02-20T12:36:09.063 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/yuriw-2024-02-19_20:21:18-fs-wip-yuri5-testing-2024-02-16-1120-quincy-distro-default-smithi/7567673 branch: wip-yuri5-testing-2024-02-16-1120-quincy description: fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/pacific 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-inline/no 4-verify} 2-client 3-upgrade-with-workload 4-verify}} email: yweinste@redhat.com first_in_suite: false job_id: '7567673' kernel: kdb: true sha1: distro last_in_suite: false machine_type: smithi meta: - desc: 'setup ceph/pacific ' name: yuriw-2024-02-19_20:21:18-fs-wip-yuri5-testing-2024-02-16-1120-quincy-distro-default-smithi no_nested_subset: false nuke-on-error: true os_type: centos os_version: 8.stream overrides: admin_socket: branch: wip-yuri5-testing-2024-02-16-1120-quincy ceph: conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: mon pg warn min per osd: 0 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - \(POOL_APP_NOT_ENABLED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding sha1: b0bd01b8442d66b8a31f8cb3050334b026de2f3b ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: b0bd01b8442d66b8a31f8cb3050334b026de2f3b kclient: syntax: v1 selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-yuri5-testing-2024-02-16-1120-quincy sha1: b0bd01b8442d66b8a31f8cb3050334b026de2f3b owner: scheduled_yuriw@teuthology priority: 71 repo: https://github.com/ceph/ceph-ci.git roles: - - host.a - client.0 - osd.0 - osd.1 - osd.2 - - host.b - client.1 - osd.3 - osd.4 - osd.5 seed: 4250 sha1: b0bd01b8442d66b8a31f8cb3050334b026de2f3b sleep_before_teardown: 0 subset: 111/120000 suite: fs suite_branch: wip-yuri5-testing-2024-02-16-1120-quincy suite_path: /home/teuthworker/src/github.com_ceph_ceph-c_b0bd01b8442d66b8a31f8cb3050334b026de2f3b/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph-ci.git suite_sha1: b0bd01b8442d66b8a31f8cb3050334b026de2f3b targets: smithi064.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBaltUs4d994y8I+1xeyo/rx86HpvBqhma4wgetKpfmdAPu4CPm4wVVKw08FRyoFJ6PLXHBKHNnadG75dXQEtqI= smithi071.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBC5alVGgGW1Ddqv3KW7HbscCa8SQnau0YCExXl3AACgrxv29h791W7mrH+v+6U9V83I/9+z8IkK189a2j34PrHc= tasks: - pexec: all: - sudo cp /etc/containers/registries.conf /etc/containers/registries.conf.backup - sudo dnf -y module reset container-tools - sudo dnf -y module install container-tools --allowerasing --nobest - sudo cp /etc/containers/registries.conf.backup /etc/containers/registries.conf - install: branch: pacific exclude_packages: - ceph-volume - print: '**** done install task...' - cephadm: cephadm_branch: pacific cephadm_git_url: https://github.com/ceph/ceph conf: osd: osd_class_default_list: '*' osd_class_load_list: '*' image: docker.io/ceph/daemon-base:latest-pacific roleless: true - print: '**** done end installing pacific cephadm ...' - cephadm.shell: host.a: - ceph config set mgr mgr/cephadm/use_repo_digest true --force - print: '**** done cephadm.shell ceph config set mgr...' - cephadm.shell: host.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: host.a: - ceph fs volume create cephfs --placement=4 - ceph fs dump - cephadm.shell: host.a: - ceph fs set cephfs max_mds 1 - cephadm.shell: host.a: - ceph fs set cephfs allow_standby_replay false - cephadm.shell: host.a: - ceph fs set cephfs inline_data false - cephadm.shell: host.a: - ceph fs dump - ceph --format=json fs dump | jq -e ".filesystems | length == 1" - while ! ceph --format=json mds versions | jq -e ". | add == 4"; do sleep 1; done - fs.pre_upgrade_save: null - kclient: null - print: '**** done client' - parallel: - upgrade-tasks - workload-tasks - cephadm.shell: host.a: - ceph fs dump - fs.post_upgrade_checks: null teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: b1dac5519c57c269ea98a22fb7729016a1d0e4d2 timestamp: 2024-02-19_20:21:18 tube: smithi upgrade-tasks: sequential: - cephadm.shell: env: - sha1 host.a: - ceph config set mon mon_warn_on_insecure_global_id_reclaim false --force - ceph config set mon mon_warn_on_insecure_global_id_reclaim_allowed false --force - ceph config set global log_to_journald false --force - ceph orch upgrade start --image quay.ceph.io/ceph-ci/ceph:$sha1 - cephadm.shell: env: - sha1 host.a: - while ceph orch upgrade status | jq '.in_progress' | grep true ; do ceph orch ps ; ceph versions ; ceph fs dump; sleep 30 ; done - ceph orch ps - ceph versions - echo "wait for servicemap items w/ changing names to refresh" - sleep 60 - ceph orch ps - ceph versions - ceph versions | jq -e '.overall | length == 1' - ceph versions | jq -e '.overall | keys' | grep $sha1 user: yuriw verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 workload-tasks: sequential: - workunit: clients: all: - suites/fsstress.sh 2024-02-20T12:36:09.063 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph-c_b0bd01b8442d66b8a31f8cb3050334b026de2f3b/qa; will attempt to use it 2024-02-20T12:36:09.063 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph-c_b0bd01b8442d66b8a31f8cb3050334b026de2f3b/qa/tasks 2024-02-20T12:36:09.064 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-02-20T12:36:09.065 INFO:teuthology.task.internal:Checking packages... 2024-02-20T12:36:09.089 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'b0bd01b8442d66b8a31f8cb3050334b026de2f3b' 2024-02-20T12:36:09.089 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-02-20T12:36:09.089 INFO:teuthology.packaging:ref: None 2024-02-20T12:36:09.089 INFO:teuthology.packaging:tag: None 2024-02-20T12:36:09.089 INFO:teuthology.packaging:branch: wip-yuri5-testing-2024-02-16-1120-quincy 2024-02-20T12:36:09.089 INFO:teuthology.packaging:sha1: b0bd01b8442d66b8a31f8cb3050334b026de2f3b 2024-02-20T12:36:09.089 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=wip-yuri5-testing-2024-02-16-1120-quincy 2024-02-20T12:36:09.374 INFO:teuthology.task.internal:Found packages for ceph version 17.2.7-543.gb0bd01b8 2024-02-20T12:36:09.376 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-02-20T12:36:09.383 INFO:teuthology.task.internal:no buildpackages task found 2024-02-20T12:36:09.383 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-02-20T12:36:09.400 INFO:teuthology.task.internal:Saving configuration 2024-02-20T12:36:09.420 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-02-20T12:36:09.425 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-02-20T12:36:09.451 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi064.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2024-02-19_20:21:18-fs-wip-yuri5-testing-2024-02-16-1120-quincy-distro-default-smithi/7567673', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '8', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-02-20 12:30:56.879902', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBaltUs4d994y8I+1xeyo/rx86HpvBqhma4wgetKpfmdAPu4CPm4wVVKw08FRyoFJ6PLXHBKHNnadG75dXQEtqI='} 2024-02-20T12:36:09.472 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi071.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2024-02-19_20:21:18-fs-wip-yuri5-testing-2024-02-16-1120-quincy-distro-default-smithi/7567673', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '8', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-02-20 12:30:56.881907', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBC5alVGgGW1Ddqv3KW7HbscCa8SQnau0YCExXl3AACgrxv29h791W7mrH+v+6U9V83I/9+z8IkK189a2j34PrHc='} 2024-02-20T12:36:09.473 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-02-20T12:36:09.478 INFO:teuthology.task.internal:roles: ubuntu@smithi064.front.sepia.ceph.com - ['host.a', 'client.0', 'osd.0', 'osd.1', 'osd.2'] 2024-02-20T12:36:09.478 INFO:teuthology.task.internal:roles: ubuntu@smithi071.front.sepia.ceph.com - ['host.b', 'client.1', 'osd.3', 'osd.4', 'osd.5'] 2024-02-20T12:36:09.479 INFO:teuthology.run_tasks:Running task console_log... 2024-02-20T12:36:09.553 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fe7ce348160>, signals=[15]) 2024-02-20T12:36:09.553 INFO:teuthology.run_tasks:Running task internal.connect... 2024-02-20T12:36:09.559 INFO:teuthology.task.internal:Opening connections... 2024-02-20T12:36:09.559 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi064.front.sepia.ceph.com 2024-02-20T12:36:09.561 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi064.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-20T12:36:09.638 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi071.front.sepia.ceph.com 2024-02-20T12:36:09.639 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi071.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-20T12:36:09.714 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-02-20T12:36:09.720 DEBUG:teuthology.orchestra.run.smithi064:> uname -m 2024-02-20T12:36:09.743 INFO:teuthology.orchestra.run.smithi064.stdout:x86_64 2024-02-20T12:36:09.743 DEBUG:teuthology.orchestra.run.smithi064:> cat /etc/os-release 2024-02-20T12:36:09.805 INFO:teuthology.orchestra.run.smithi064.stdout:NAME="CentOS Stream" 2024-02-20T12:36:09.805 INFO:teuthology.orchestra.run.smithi064.stdout:VERSION="8" 2024-02-20T12:36:09.805 INFO:teuthology.orchestra.run.smithi064.stdout:ID="centos" 2024-02-20T12:36:09.806 INFO:teuthology.orchestra.run.smithi064.stdout:ID_LIKE="rhel fedora" 2024-02-20T12:36:09.806 INFO:teuthology.orchestra.run.smithi064.stdout:VERSION_ID="8" 2024-02-20T12:36:09.806 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM_ID="platform:el8" 2024-02-20T12:36:09.806 INFO:teuthology.orchestra.run.smithi064.stdout:PRETTY_NAME="CentOS Stream 8" 2024-02-20T12:36:09.806 INFO:teuthology.orchestra.run.smithi064.stdout:ANSI_COLOR="0;31" 2024-02-20T12:36:09.806 INFO:teuthology.orchestra.run.smithi064.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2024-02-20T12:36:09.806 INFO:teuthology.orchestra.run.smithi064.stdout:HOME_URL="https://centos.org/" 2024-02-20T12:36:09.806 INFO:teuthology.orchestra.run.smithi064.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-02-20T12:36:09.806 INFO:teuthology.orchestra.run.smithi064.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2024-02-20T12:36:09.806 INFO:teuthology.orchestra.run.smithi064.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-02-20T12:36:09.807 INFO:teuthology.lock.ops:Updating smithi064.front.sepia.ceph.com on lock server 2024-02-20T12:36:09.826 DEBUG:teuthology.orchestra.run.smithi071:> uname -m 2024-02-20T12:36:09.848 INFO:teuthology.orchestra.run.smithi071.stdout:x86_64 2024-02-20T12:36:09.848 DEBUG:teuthology.orchestra.run.smithi071:> cat /etc/os-release 2024-02-20T12:36:09.909 INFO:teuthology.orchestra.run.smithi071.stdout:NAME="CentOS Stream" 2024-02-20T12:36:09.909 INFO:teuthology.orchestra.run.smithi071.stdout:VERSION="8" 2024-02-20T12:36:09.909 INFO:teuthology.orchestra.run.smithi071.stdout:ID="centos" 2024-02-20T12:36:09.909 INFO:teuthology.orchestra.run.smithi071.stdout:ID_LIKE="rhel fedora" 2024-02-20T12:36:09.909 INFO:teuthology.orchestra.run.smithi071.stdout:VERSION_ID="8" 2024-02-20T12:36:09.909 INFO:teuthology.orchestra.run.smithi071.stdout:PLATFORM_ID="platform:el8" 2024-02-20T12:36:09.909 INFO:teuthology.orchestra.run.smithi071.stdout:PRETTY_NAME="CentOS Stream 8" 2024-02-20T12:36:09.909 INFO:teuthology.orchestra.run.smithi071.stdout:ANSI_COLOR="0;31" 2024-02-20T12:36:09.910 INFO:teuthology.orchestra.run.smithi071.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2024-02-20T12:36:09.910 INFO:teuthology.orchestra.run.smithi071.stdout:HOME_URL="https://centos.org/" 2024-02-20T12:36:09.910 INFO:teuthology.orchestra.run.smithi071.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-02-20T12:36:09.910 INFO:teuthology.orchestra.run.smithi071.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2024-02-20T12:36:09.910 INFO:teuthology.orchestra.run.smithi071.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-02-20T12:36:09.910 INFO:teuthology.lock.ops:Updating smithi071.front.sepia.ceph.com on lock server 2024-02-20T12:36:09.934 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-02-20T12:36:09.943 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-02-20T12:36:09.948 INFO:teuthology.task.internal:Checking for old test directory... 2024-02-20T12:36:09.948 DEBUG:teuthology.orchestra.run.smithi064:> test '!' -e /home/ubuntu/cephtest 2024-02-20T12:36:09.951 DEBUG:teuthology.orchestra.run.smithi071:> test '!' -e /home/ubuntu/cephtest 2024-02-20T12:36:09.970 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-02-20T12:36:09.975 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-02-20T12:36:09.975 DEBUG:teuthology.orchestra.run.smithi064:> test -z $(ls -A /var/lib/ceph) 2024-02-20T12:36:10.012 DEBUG:teuthology.orchestra.run.smithi071:> test -z $(ls -A /var/lib/ceph) 2024-02-20T12:36:10.041 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-02-20T12:36:10.097 INFO:teuthology.run_tasks:Running task kernel... 2024-02-20T12:36:10.111 INFO:teuthology.task.kernel:normalize config orig: {'kdb': True, 'sha1': 'distro'} 2024-02-20T12:36:10.111 INFO:teuthology.task.kernel:config {'host.a': {'kdb': True, 'sha1': 'distro'}, 'host.b': {'kdb': True, 'sha1': 'distro'}}, timeout 300 2024-02-20T12:36:10.112 DEBUG:teuthology.orchestra.run.smithi064:> test -f /run/.containerenv -o -f /.dockerenv 2024-02-20T12:36:10.112 DEBUG:teuthology.orchestra.run.smithi071:> test -f /run/.containerenv -o -f /.dockerenv 2024-02-20T12:36:10.132 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-20T12:36:10.133 DEBUG:teuthology.orchestra.run.smithi071:> uname -r 2024-02-20T12:36:10.134 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-20T12:36:10.135 DEBUG:teuthology.orchestra.run.smithi064:> uname -r 2024-02-20T12:36:10.193 INFO:teuthology.orchestra.run.smithi071.stdout:4.18.0-489.el8.x86_64 2024-02-20T12:36:10.193 INFO:teuthology.task.kernel:Running kernel on smithi071: 4.18.0-489.el8.x86_64 2024-02-20T12:36:10.194 DEBUG:teuthology.orchestra.run.smithi071:> sudo yum install -y kernel 2024-02-20T12:36:10.196 INFO:teuthology.orchestra.run.smithi064.stdout:4.18.0-489.el8.x86_64 2024-02-20T12:36:10.197 INFO:teuthology.task.kernel:Running kernel on smithi064: 4.18.0-489.el8.x86_64 2024-02-20T12:36:10.197 DEBUG:teuthology.orchestra.run.smithi064:> sudo yum install -y kernel 2024-02-20T12:36:14.790 INFO:teuthology.orchestra.run.smithi064.stdout:CentOS Stream 8 - AppStream 14 MB/s | 27 MB 00:01 2024-02-20T12:36:16.297 INFO:teuthology.orchestra.run.smithi071.stdout:CentOS Stream 8 - AppStream 7.6 MB/s | 27 MB 00:03 2024-02-20T12:36:20.507 INFO:teuthology.orchestra.run.smithi064.stdout:CentOS Stream 8 - BaseOS 17 MB/s | 10 MB 00:00 2024-02-20T12:36:21.581 INFO:teuthology.orchestra.run.smithi071.stdout:CentOS Stream 8 - BaseOS 23 MB/s | 10 MB 00:00 2024-02-20T12:36:22.532 INFO:teuthology.orchestra.run.smithi064.stdout:CentOS Stream 8 - Extras 60 kB/s | 18 kB 00:00 2024-02-20T12:36:23.100 INFO:teuthology.orchestra.run.smithi064.stdout:CentOS Stream 8 - Extras common packages 15 kB/s | 6.9 kB 00:00 2024-02-20T12:36:23.359 INFO:teuthology.orchestra.run.smithi064.stdout:Copr repo for python3-asyncssh owned by ceph 22 kB/s | 3.5 kB 00:00 2024-02-20T12:36:23.472 INFO:teuthology.orchestra.run.smithi071.stdout:CentOS Stream 8 - Extras 64 kB/s | 18 kB 00:00 2024-02-20T12:36:24.014 INFO:teuthology.orchestra.run.smithi071.stdout:CentOS Stream 8 - Extras common packages 16 kB/s | 6.9 kB 00:00 2024-02-20T12:36:24.299 INFO:teuthology.orchestra.run.smithi071.stdout:Copr repo for python3-asyncssh owned by ceph 18 kB/s | 3.5 kB 00:00 2024-02-20T12:36:25.992 INFO:teuthology.orchestra.run.smithi071.stdout:Extra Packages for Enterprise Linux 10 MB/s | 16 MB 00:01 2024-02-20T12:36:29.817 INFO:teuthology.orchestra.run.smithi071.stdout:lab-extras 423 kB/s | 24 kB 00:00 2024-02-20T12:36:32.813 INFO:teuthology.orchestra.run.smithi071.stdout:Package kernel-4.18.0-348.el8.x86_64 is already installed. 2024-02-20T12:36:32.813 INFO:teuthology.orchestra.run.smithi071.stdout:Package kernel-4.18.0-489.el8.x86_64 is already installed. 2024-02-20T12:36:32.917 INFO:teuthology.orchestra.run.smithi071.stdout:Dependencies resolved. 2024-02-20T12:36:32.918 INFO:teuthology.orchestra.run.smithi071.stdout:================================================================================ 2024-02-20T12:36:32.918 INFO:teuthology.orchestra.run.smithi071.stdout: Package Architecture Version Repository Size 2024-02-20T12:36:32.918 INFO:teuthology.orchestra.run.smithi071.stdout:================================================================================ 2024-02-20T12:36:32.918 INFO:teuthology.orchestra.run.smithi071.stdout:Installing: 2024-02-20T12:36:32.918 INFO:teuthology.orchestra.run.smithi071.stdout: kernel x86_64 4.18.0-539.el8 baseos 10 M 2024-02-20T12:36:32.918 INFO:teuthology.orchestra.run.smithi071.stdout:Installing dependencies: 2024-02-20T12:36:32.918 INFO:teuthology.orchestra.run.smithi071.stdout: kernel-core x86_64 4.18.0-539.el8 baseos 43 M 2024-02-20T12:36:32.918 INFO:teuthology.orchestra.run.smithi071.stdout: kernel-modules x86_64 4.18.0-539.el8 baseos 36 M 2024-02-20T12:36:32.918 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-20T12:36:32.918 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction Summary 2024-02-20T12:36:32.918 INFO:teuthology.orchestra.run.smithi071.stdout:================================================================================ 2024-02-20T12:36:32.919 INFO:teuthology.orchestra.run.smithi071.stdout:Install 3 Packages 2024-02-20T12:36:32.919 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-20T12:36:32.919 INFO:teuthology.orchestra.run.smithi071.stdout:Total download size: 90 M 2024-02-20T12:36:32.919 INFO:teuthology.orchestra.run.smithi071.stdout:Installed size: 97 M 2024-02-20T12:36:32.919 INFO:teuthology.orchestra.run.smithi071.stdout:Downloading Packages: 2024-02-20T12:36:33.459 INFO:teuthology.orchestra.run.smithi064.stdout:Extra Packages for Enterprise Linux 1.6 MB/s | 16 MB 00:10 2024-02-20T12:36:33.639 INFO:teuthology.orchestra.run.smithi071.stdout:(1/3): kernel-4.18.0-539.el8.x86_64.rpm 19 MB/s | 10 MB 00:00 2024-02-20T12:36:34.635 INFO:teuthology.orchestra.run.smithi071.stdout:(2/3): kernel-modules-4.18.0-539.el8.x86_64.rpm 24 MB/s | 36 MB 00:01 2024-02-20T12:36:35.119 INFO:teuthology.orchestra.run.smithi071.stdout:(3/3): kernel-core-4.18.0-539.el8.x86_64.rpm 21 MB/s | 43 MB 00:02 2024-02-20T12:36:35.119 INFO:teuthology.orchestra.run.smithi071.stdout:-------------------------------------------------------------------------------- 2024-02-20T12:36:35.119 INFO:teuthology.orchestra.run.smithi071.stdout:Total 41 MB/s | 90 MB 00:02 2024-02-20T12:36:35.806 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction check 2024-02-20T12:36:36.222 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction check succeeded. 2024-02-20T12:36:36.222 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction test 2024-02-20T12:36:37.459 INFO:teuthology.orchestra.run.smithi064.stdout:lab-extras 558 kB/s | 24 kB 00:00 2024-02-20T12:36:37.478 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction test succeeded. 2024-02-20T12:36:37.497 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction 2024-02-20T12:36:40.298 INFO:teuthology.orchestra.run.smithi071.stdout: Preparing : 1/1 2024-02-20T12:36:40.539 INFO:teuthology.orchestra.run.smithi064.stdout:Package kernel-4.18.0-348.el8.x86_64 is already installed. 2024-02-20T12:36:40.540 INFO:teuthology.orchestra.run.smithi064.stdout:Package kernel-4.18.0-489.el8.x86_64 is already installed. 2024-02-20T12:36:40.653 INFO:teuthology.orchestra.run.smithi064.stdout:Dependencies resolved. 2024-02-20T12:36:40.654 INFO:teuthology.orchestra.run.smithi064.stdout:================================================================================ 2024-02-20T12:36:40.654 INFO:teuthology.orchestra.run.smithi064.stdout: Package Architecture Version Repository Size 2024-02-20T12:36:40.654 INFO:teuthology.orchestra.run.smithi064.stdout:================================================================================ 2024-02-20T12:36:40.655 INFO:teuthology.orchestra.run.smithi064.stdout:Installing: 2024-02-20T12:36:40.655 INFO:teuthology.orchestra.run.smithi064.stdout: kernel x86_64 4.18.0-539.el8 baseos 10 M 2024-02-20T12:36:40.655 INFO:teuthology.orchestra.run.smithi064.stdout:Installing dependencies: 2024-02-20T12:36:40.655 INFO:teuthology.orchestra.run.smithi064.stdout: kernel-core x86_64 4.18.0-539.el8 baseos 43 M 2024-02-20T12:36:40.655 INFO:teuthology.orchestra.run.smithi064.stdout: kernel-modules x86_64 4.18.0-539.el8 baseos 36 M 2024-02-20T12:36:40.655 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-20T12:36:40.655 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction Summary 2024-02-20T12:36:40.655 INFO:teuthology.orchestra.run.smithi064.stdout:================================================================================ 2024-02-20T12:36:40.655 INFO:teuthology.orchestra.run.smithi064.stdout:Install 3 Packages 2024-02-20T12:36:40.655 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-20T12:36:40.656 INFO:teuthology.orchestra.run.smithi064.stdout:Total download size: 90 M 2024-02-20T12:36:40.656 INFO:teuthology.orchestra.run.smithi064.stdout:Installed size: 97 M 2024-02-20T12:36:40.656 INFO:teuthology.orchestra.run.smithi064.stdout:Downloading Packages: 2024-02-20T12:36:40.842 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : kernel-core-4.18.0-539.el8.x86_64 1/3 2024-02-20T12:36:41.269 INFO:teuthology.orchestra.run.smithi064.stdout:(1/3): kernel-4.18.0-539.el8.x86_64.rpm 24 MB/s | 10 MB 00:00 2024-02-20T12:36:42.161 INFO:teuthology.orchestra.run.smithi064.stdout:(2/3): kernel-modules-4.18.0-539.el8.x86_64.rpm 27 MB/s | 36 MB 00:01 2024-02-20T12:36:42.619 INFO:teuthology.orchestra.run.smithi064.stdout:(3/3): kernel-core-4.18.0-539.el8.x86_64.rpm 24 MB/s | 43 MB 00:01 2024-02-20T12:36:42.619 INFO:teuthology.orchestra.run.smithi064.stdout:-------------------------------------------------------------------------------- 2024-02-20T12:36:42.620 INFO:teuthology.orchestra.run.smithi064.stdout:Total 46 MB/s | 90 MB 00:01 2024-02-20T12:36:43.332 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction check 2024-02-20T12:36:43.802 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction check succeeded. 2024-02-20T12:36:43.803 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction test 2024-02-20T12:36:44.401 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: kernel-core-4.18.0-539.el8.x86_64 1/3 2024-02-20T12:36:44.952 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : kernel-modules-4.18.0-539.el8.x86_64 2/3 2024-02-20T12:36:45.108 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction test succeeded. 2024-02-20T12:36:45.135 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction 2024-02-20T12:36:48.096 INFO:teuthology.orchestra.run.smithi064.stdout: Preparing : 1/1 2024-02-20T12:36:48.384 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: kernel-modules-4.18.0-539.el8.x86_64 2/3 2024-02-20T12:36:48.632 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : kernel-core-4.18.0-539.el8.x86_64 1/3 2024-02-20T12:36:48.672 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : kernel-4.18.0-539.el8.x86_64 3/3 2024-02-20T12:36:52.587 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: kernel-core-4.18.0-539.el8.x86_64 1/3 2024-02-20T12:36:52.946 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : kernel-modules-4.18.0-539.el8.x86_64 2/3 2024-02-20T12:36:56.627 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: kernel-modules-4.18.0-539.el8.x86_64 2/3 2024-02-20T12:36:56.931 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : kernel-4.18.0-539.el8.x86_64 3/3 2024-02-20T12:37:10.795 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: kernel-core-4.18.0-539.el8.x86_64 3/3 2024-02-20T12:37:10.818 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: kernel-modules-4.18.0-539.el8.x86_64 3/3 2024-02-20T12:37:11.168 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: kernel-4.18.0-539.el8.x86_64 3/3 2024-02-20T12:37:11.168 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : kernel-4.18.0-539.el8.x86_64 1/3 2024-02-20T12:37:11.169 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : kernel-core-4.18.0-539.el8.x86_64 2/3 2024-02-20T12:37:11.823 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : kernel-modules-4.18.0-539.el8.x86_64 3/3 2024-02-20T12:37:11.823 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-20T12:37:11.823 INFO:teuthology.orchestra.run.smithi071.stdout:Installed: 2024-02-20T12:37:11.823 INFO:teuthology.orchestra.run.smithi071.stdout: kernel-4.18.0-539.el8.x86_64 kernel-core-4.18.0-539.el8.x86_64 2024-02-20T12:37:11.823 INFO:teuthology.orchestra.run.smithi071.stdout: kernel-modules-4.18.0-539.el8.x86_64 2024-02-20T12:37:11.823 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-20T12:37:11.823 INFO:teuthology.orchestra.run.smithi071.stdout:Complete! 2024-02-20T12:37:11.966 DEBUG:teuthology.orchestra.run.smithi071:> rpm -q kernel | sort -rV | head -n 1 2024-02-20T12:37:12.204 INFO:teuthology.orchestra.run.smithi071.stdout:kernel-4.18.0-539.el8.x86_64 2024-02-20T12:37:12.205 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-539.el8.x86_64 2024-02-20T12:37:12.205 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 4.18.0-489.el8.x86_64 Expected: 4.18.0-539.el8.x86_64 2024-02-20T12:37:12.205 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-02-20T12:37:12.205 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-02-20T12:37:12.205 INFO:teuthology.task.kernel:Installing distro kernel on host.b... 2024-02-20T12:37:12.205 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi071.front.sepia.ceph.com, path=None, version=distro) 2024-02-20T12:37:12.205 DEBUG:teuthology.orchestra.run.smithi071:> rpm -q kernel | sort -rV | head -n 1 2024-02-20T12:37:12.414 INFO:teuthology.orchestra.run.smithi071.stdout:kernel-4.18.0-539.el8.x86_64 2024-02-20T12:37:12.444 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-539.el8.x86_64 2024-02-20T12:37:12.444 DEBUG:teuthology.orchestra.run.smithi071:> sudo rpm -qi grub2-tools 2024-02-20T12:37:12.479 INFO:teuthology.orchestra.run.smithi071.stdout:Name : grub2-tools 2024-02-20T12:37:12.479 INFO:teuthology.orchestra.run.smithi071.stdout:Epoch : 1 2024-02-20T12:37:12.479 INFO:teuthology.orchestra.run.smithi071.stdout:Version : 2.02 2024-02-20T12:37:12.479 INFO:teuthology.orchestra.run.smithi071.stdout:Release : 129.el8 2024-02-20T12:37:12.479 INFO:teuthology.orchestra.run.smithi071.stdout:Architecture: x86_64 2024-02-20T12:37:12.479 INFO:teuthology.orchestra.run.smithi071.stdout:Install Date: Thu 04 May 2023 10:17:48 PM UTC 2024-02-20T12:37:12.480 INFO:teuthology.orchestra.run.smithi071.stdout:Group : System Environment/Base 2024-02-20T12:37:12.480 INFO:teuthology.orchestra.run.smithi071.stdout:Size : 9265586 2024-02-20T12:37:12.480 INFO:teuthology.orchestra.run.smithi071.stdout:License : GPLv3+ 2024-02-20T12:37:12.480 INFO:teuthology.orchestra.run.smithi071.stdout:Signature : RSA/SHA256, Tue 19 Jul 2022 02:52:06 PM UTC, Key ID 05b555b38483c65d 2024-02-20T12:37:12.480 INFO:teuthology.orchestra.run.smithi071.stdout:Source RPM : grub2-2.02-129.el8.src.rpm 2024-02-20T12:37:12.480 INFO:teuthology.orchestra.run.smithi071.stdout:Build Date : Tue 19 Jul 2022 02:30:39 PM UTC 2024-02-20T12:37:12.480 INFO:teuthology.orchestra.run.smithi071.stdout:Build Host : kbuilder.bsys.centos.org 2024-02-20T12:37:12.481 INFO:teuthology.orchestra.run.smithi071.stdout:Relocations : (not relocatable) 2024-02-20T12:37:12.481 INFO:teuthology.orchestra.run.smithi071.stdout:Packager : CentOS BuildSystem 2024-02-20T12:37:12.481 INFO:teuthology.orchestra.run.smithi071.stdout:Vendor : CentOS 2024-02-20T12:37:12.481 INFO:teuthology.orchestra.run.smithi071.stdout:URL : http://www.gnu.org/software/grub/ 2024-02-20T12:37:12.481 INFO:teuthology.orchestra.run.smithi071.stdout:Summary : Support tools for GRUB. 2024-02-20T12:37:12.481 INFO:teuthology.orchestra.run.smithi071.stdout:Description : 2024-02-20T12:37:12.481 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-20T12:37:12.482 INFO:teuthology.orchestra.run.smithi071.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-02-20T12:37:12.482 INFO:teuthology.orchestra.run.smithi071.stdout:customizable bootloader with modular architecture. It supports a rich 2024-02-20T12:37:12.482 INFO:teuthology.orchestra.run.smithi071.stdout:variety of kernel formats, file systems, computer architectures and 2024-02-20T12:37:12.482 INFO:teuthology.orchestra.run.smithi071.stdout:hardware devices. 2024-02-20T12:37:12.482 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-20T12:37:12.482 INFO:teuthology.orchestra.run.smithi071.stdout:This subpackage provides tools for support of all platforms. 2024-02-20T12:37:12.484 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-02-20T12:37:12.484 INFO:teuthology.task.kernel:Updating grub on smithi071 to boot 4.18.0-539.el8.x86_64 2024-02-20T12:37:12.484 DEBUG:teuthology.orchestra.run.smithi071:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-02-20T12:37:14.058 INFO:teuthology.orchestra.run.smithi071.stderr:Generating grub configuration file ... 2024-02-20T12:37:16.543 INFO:teuthology.orchestra.run.smithi071.stderr:done 2024-02-20T12:37:16.546 DEBUG:teuthology.orchestra.run.smithi071:> mktemp 2024-02-20T12:37:16.568 INFO:teuthology.orchestra.run.smithi071.stdout:/tmp/tmp.qUTd99QZRK 2024-02-20T12:37:16.568 DEBUG:teuthology.orchestra.run.smithi071:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.qUTd99QZRK 2024-02-20T12:37:16.641 DEBUG:teuthology.orchestra.run.smithi071:> sudo chmod 0666 /tmp/tmp.qUTd99QZRK 2024-02-20T12:37:16.789 DEBUG:teuthology.orchestra.remote:smithi071:/tmp/tmp.qUTd99QZRK is 6KB 2024-02-20T12:37:16.838 DEBUG:teuthology.orchestra.run.smithi071:> rm -fr /tmp/tmp.qUTd99QZRK 2024-02-20T12:37:16.855 DEBUG:teuthology.orchestra.run.smithi071:> sudo /bin/ls /boot/loader/entries || true 2024-02-20T12:37:16.923 INFO:teuthology.orchestra.run.smithi071.stdout:a1b29d6c289747bdb22930598d39874a-0-rescue.conf 2024-02-20T12:37:16.923 INFO:teuthology.orchestra.run.smithi071.stdout:a1b29d6c289747bdb22930598d39874a-4.18.0-348.el8.x86_64.conf 2024-02-20T12:37:16.923 INFO:teuthology.orchestra.run.smithi071.stdout:a1b29d6c289747bdb22930598d39874a-4.18.0-489.el8.x86_64.conf 2024-02-20T12:37:16.923 INFO:teuthology.orchestra.run.smithi071.stdout:a1b29d6c289747bdb22930598d39874a-4.18.0-539.el8.x86_64.conf 2024-02-20T12:37:16.923 DEBUG:teuthology.orchestra.run.smithi071:> sudo grub2-set-default a1b29d6c289747bdb22930598d39874a-4.18.0-539.el8.x86_64 2024-02-20T12:37:17.127 DEBUG:teuthology.orchestra.run.smithi071:> sudo shutdown -r now 2024-02-20T12:37:20.305 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: kernel-core-4.18.0-539.el8.x86_64 3/3 2024-02-20T12:37:20.329 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: kernel-modules-4.18.0-539.el8.x86_64 3/3 2024-02-20T12:37:20.698 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: kernel-4.18.0-539.el8.x86_64 3/3 2024-02-20T12:37:20.698 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : kernel-4.18.0-539.el8.x86_64 1/3 2024-02-20T12:37:20.698 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : kernel-core-4.18.0-539.el8.x86_64 2/3 2024-02-20T12:37:21.517 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : kernel-modules-4.18.0-539.el8.x86_64 3/3 2024-02-20T12:37:21.517 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-20T12:37:21.517 INFO:teuthology.orchestra.run.smithi064.stdout:Installed: 2024-02-20T12:37:21.517 INFO:teuthology.orchestra.run.smithi064.stdout: kernel-4.18.0-539.el8.x86_64 kernel-core-4.18.0-539.el8.x86_64 2024-02-20T12:37:21.517 INFO:teuthology.orchestra.run.smithi064.stdout: kernel-modules-4.18.0-539.el8.x86_64 2024-02-20T12:37:21.517 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-20T12:37:21.517 INFO:teuthology.orchestra.run.smithi064.stdout:Complete! 2024-02-20T12:37:21.654 DEBUG:teuthology.orchestra.run.smithi064:> rpm -q kernel | sort -rV | head -n 1 2024-02-20T12:37:21.921 INFO:teuthology.orchestra.run.smithi064.stdout:kernel-4.18.0-539.el8.x86_64 2024-02-20T12:37:21.921 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-539.el8.x86_64 2024-02-20T12:37:21.921 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 4.18.0-489.el8.x86_64 Expected: 4.18.0-539.el8.x86_64 2024-02-20T12:37:21.921 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-02-20T12:37:21.921 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-02-20T12:37:21.922 INFO:teuthology.task.kernel:Installing distro kernel on host.a... 2024-02-20T12:37:21.922 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi064.front.sepia.ceph.com, path=None, version=distro) 2024-02-20T12:37:21.922 DEBUG:teuthology.orchestra.run.smithi064:> rpm -q kernel | sort -rV | head -n 1 2024-02-20T12:37:22.141 INFO:teuthology.orchestra.run.smithi064.stdout:kernel-4.18.0-539.el8.x86_64 2024-02-20T12:37:22.142 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-539.el8.x86_64 2024-02-20T12:37:22.142 DEBUG:teuthology.orchestra.run.smithi064:> sudo rpm -qi grub2-tools 2024-02-20T12:37:22.176 INFO:teuthology.orchestra.run.smithi064.stdout:Name : grub2-tools 2024-02-20T12:37:22.176 INFO:teuthology.orchestra.run.smithi064.stdout:Epoch : 1 2024-02-20T12:37:22.176 INFO:teuthology.orchestra.run.smithi064.stdout:Version : 2.02 2024-02-20T12:37:22.176 INFO:teuthology.orchestra.run.smithi064.stdout:Release : 129.el8 2024-02-20T12:37:22.176 INFO:teuthology.orchestra.run.smithi064.stdout:Architecture: x86_64 2024-02-20T12:37:22.176 INFO:teuthology.orchestra.run.smithi064.stdout:Install Date: Thu 04 May 2023 10:17:48 PM UTC 2024-02-20T12:37:22.176 INFO:teuthology.orchestra.run.smithi064.stdout:Group : System Environment/Base 2024-02-20T12:37:22.176 INFO:teuthology.orchestra.run.smithi064.stdout:Size : 9265586 2024-02-20T12:37:22.176 INFO:teuthology.orchestra.run.smithi064.stdout:License : GPLv3+ 2024-02-20T12:37:22.177 INFO:teuthology.orchestra.run.smithi064.stdout:Signature : RSA/SHA256, Tue 19 Jul 2022 02:52:06 PM UTC, Key ID 05b555b38483c65d 2024-02-20T12:37:22.177 INFO:teuthology.orchestra.run.smithi064.stdout:Source RPM : grub2-2.02-129.el8.src.rpm 2024-02-20T12:37:22.177 INFO:teuthology.orchestra.run.smithi064.stdout:Build Date : Tue 19 Jul 2022 02:30:39 PM UTC 2024-02-20T12:37:22.177 INFO:teuthology.orchestra.run.smithi064.stdout:Build Host : kbuilder.bsys.centos.org 2024-02-20T12:37:22.177 INFO:teuthology.orchestra.run.smithi064.stdout:Relocations : (not relocatable) 2024-02-20T12:37:22.177 INFO:teuthology.orchestra.run.smithi064.stdout:Packager : CentOS BuildSystem 2024-02-20T12:37:22.177 INFO:teuthology.orchestra.run.smithi064.stdout:Vendor : CentOS 2024-02-20T12:37:22.177 INFO:teuthology.orchestra.run.smithi064.stdout:URL : http://www.gnu.org/software/grub/ 2024-02-20T12:37:22.177 INFO:teuthology.orchestra.run.smithi064.stdout:Summary : Support tools for GRUB. 2024-02-20T12:37:22.177 INFO:teuthology.orchestra.run.smithi064.stdout:Description : 2024-02-20T12:37:22.177 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-20T12:37:22.177 INFO:teuthology.orchestra.run.smithi064.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-02-20T12:37:22.177 INFO:teuthology.orchestra.run.smithi064.stdout:customizable bootloader with modular architecture. It supports a rich 2024-02-20T12:37:22.178 INFO:teuthology.orchestra.run.smithi064.stdout:variety of kernel formats, file systems, computer architectures and 2024-02-20T12:37:22.178 INFO:teuthology.orchestra.run.smithi064.stdout:hardware devices. 2024-02-20T12:37:22.178 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-20T12:37:22.178 INFO:teuthology.orchestra.run.smithi064.stdout:This subpackage provides tools for support of all platforms. 2024-02-20T12:37:22.179 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-02-20T12:37:22.179 INFO:teuthology.task.kernel:Updating grub on smithi064 to boot 4.18.0-539.el8.x86_64 2024-02-20T12:37:22.179 DEBUG:teuthology.orchestra.run.smithi064:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-02-20T12:37:23.486 INFO:teuthology.orchestra.run.smithi064.stderr:Generating grub configuration file ... 2024-02-20T12:37:25.650 INFO:teuthology.orchestra.run.smithi064.stderr:done 2024-02-20T12:37:25.651 DEBUG:teuthology.orchestra.run.smithi064:> mktemp 2024-02-20T12:37:25.673 INFO:teuthology.orchestra.run.smithi064.stdout:/tmp/tmp.XiYtSp8c1g 2024-02-20T12:37:25.676 DEBUG:teuthology.orchestra.run.smithi064:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.XiYtSp8c1g 2024-02-20T12:37:25.746 DEBUG:teuthology.orchestra.run.smithi064:> sudo chmod 0666 /tmp/tmp.XiYtSp8c1g 2024-02-20T12:37:25.885 DEBUG:teuthology.orchestra.remote:smithi064:/tmp/tmp.XiYtSp8c1g is 6KB 2024-02-20T12:37:25.933 DEBUG:teuthology.orchestra.run.smithi064:> rm -fr /tmp/tmp.XiYtSp8c1g 2024-02-20T12:37:25.951 DEBUG:teuthology.orchestra.run.smithi064:> sudo /bin/ls /boot/loader/entries || true 2024-02-20T12:37:26.026 INFO:teuthology.orchestra.run.smithi064.stdout:a1b29d6c289747bdb22930598d39874a-0-rescue.conf 2024-02-20T12:37:26.026 INFO:teuthology.orchestra.run.smithi064.stdout:a1b29d6c289747bdb22930598d39874a-4.18.0-348.el8.x86_64.conf 2024-02-20T12:37:26.026 INFO:teuthology.orchestra.run.smithi064.stdout:a1b29d6c289747bdb22930598d39874a-4.18.0-489.el8.x86_64.conf 2024-02-20T12:37:26.026 INFO:teuthology.orchestra.run.smithi064.stdout:a1b29d6c289747bdb22930598d39874a-4.18.0-539.el8.x86_64.conf 2024-02-20T12:37:26.028 DEBUG:teuthology.orchestra.run.smithi064:> sudo grub2-set-default a1b29d6c289747bdb22930598d39874a-4.18.0-539.el8.x86_64 2024-02-20T12:37:26.187 DEBUG:teuthology.orchestra.run.smithi064:> sudo shutdown -r now 2024-02-20T12:37:47.151 INFO:teuthology.task.kernel:Checking client host.b for new kernel version... 2024-02-20T12:37:47.152 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi071.front.sepia.ceph.com' 2024-02-20T12:37:47.153 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi071.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-20T12:37:56.199 INFO:teuthology.task.kernel:Checking client host.a for new kernel version... 2024-02-20T12:37:56.199 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi064.front.sepia.ceph.com' 2024-02-20T12:37:56.200 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi064.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-20T12:38:05.628 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.71 2024-02-20T12:38:14.592 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.64 2024-02-20T12:38:14.629 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi071.front.sepia.ceph.com' 2024-02-20T12:38:14.630 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi071.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-20T12:38:17.692 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.71 2024-02-20T12:38:23.596 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi064.front.sepia.ceph.com' 2024-02-20T12:38:23.596 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi064.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-20T12:38:26.652 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.64 2024-02-20T12:38:29.694 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi071.front.sepia.ceph.com' 2024-02-20T12:38:29.695 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi071.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-20T12:38:38.657 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi064.front.sepia.ceph.com' 2024-02-20T12:38:38.658 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi064.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-20T12:38:57.084 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.64 2024-02-20T12:39:01.602 DEBUG:teuthology.orchestra.run.smithi071:> true 2024-02-20T12:39:01.850 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi071.front.sepia.ceph.com' 2024-02-20T12:39:01.850 INFO:teuthology.task.kernel:Checking kernel version of host.b, want "4.18.0-539.el8.x86_64"... 2024-02-20T12:39:01.851 DEBUG:teuthology.orchestra.run.smithi071:> uname -r 2024-02-20T12:39:01.867 INFO:teuthology.orchestra.run.smithi071.stdout:4.18.0-539.el8.x86_64 2024-02-20T12:39:01.868 DEBUG:teuthology.task.kernel:current kernel version is 4.18.0-539.el8.x86_64 vs 4.18.0-539.el8.x86_64 2024-02-20T12:39:01.868 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-02-20T12:39:01.868 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-02-20T12:39:02.869 INFO:teuthology.task.kernel:Enabling kdb on host.b... 2024-02-20T12:39:02.869 DEBUG:teuthology.orchestra.run.smithi071:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-02-20T12:39:02.926 INFO:teuthology.orchestra.run.smithi071.stdout:ttyS1 2024-02-20T12:39:02.938 DEBUG:teuthology.parallel:result is None 2024-02-20T12:39:12.085 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi064.front.sepia.ceph.com' 2024-02-20T12:39:12.086 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi064.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-20T12:39:12.508 DEBUG:teuthology.orchestra.run.smithi064:> true 2024-02-20T12:39:12.767 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi064.front.sepia.ceph.com' 2024-02-20T12:39:12.768 INFO:teuthology.task.kernel:Checking kernel version of host.a, want "4.18.0-539.el8.x86_64"... 2024-02-20T12:39:12.768 DEBUG:teuthology.orchestra.run.smithi064:> uname -r 2024-02-20T12:39:12.825 INFO:teuthology.orchestra.run.smithi064.stdout:4.18.0-539.el8.x86_64 2024-02-20T12:39:12.826 DEBUG:teuthology.task.kernel:current kernel version is 4.18.0-539.el8.x86_64 vs 4.18.0-539.el8.x86_64 2024-02-20T12:39:12.826 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-02-20T12:39:12.826 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-02-20T12:39:13.827 INFO:teuthology.task.kernel:Enabling kdb on host.a... 2024-02-20T12:39:13.827 DEBUG:teuthology.orchestra.run.smithi064:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-02-20T12:39:14.601 INFO:teuthology.orchestra.run.smithi064.stdout:ttyS1 2024-02-20T12:39:14.628 DEBUG:teuthology.parallel:result is None 2024-02-20T12:39:14.628 INFO:teuthology.run_tasks:Running task internal.base... 2024-02-20T12:39:14.636 INFO:teuthology.task.internal:Creating test directory... 2024-02-20T12:39:14.636 DEBUG:teuthology.orchestra.run.smithi064:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-02-20T12:39:14.664 DEBUG:teuthology.orchestra.run.smithi071:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-02-20T12:39:15.068 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-02-20T12:39:15.075 INFO:teuthology.run_tasks:Running task internal.archive... 2024-02-20T12:39:15.083 INFO:teuthology.task.internal:Creating archive directory... 2024-02-20T12:39:15.083 DEBUG:teuthology.orchestra.run.smithi064:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-02-20T12:39:15.086 DEBUG:teuthology.orchestra.run.smithi071:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-02-20T12:39:15.141 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-02-20T12:39:15.148 INFO:teuthology.task.internal:Enabling coredump saving... 2024-02-20T12:39:15.148 DEBUG:teuthology.orchestra.run.smithi064:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-02-20T12:39:15.150 DEBUG:teuthology.orchestra.run.smithi071:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-02-20T12:39:15.179 INFO:teuthology.orchestra.run.smithi064.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-02-20T12:39:15.191 INFO:teuthology.orchestra.run.smithi064.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-02-20T12:39:15.198 INFO:teuthology.orchestra.run.smithi071.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-02-20T12:39:15.209 INFO:teuthology.orchestra.run.smithi071.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-02-20T12:39:15.210 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-02-20T12:39:15.217 INFO:teuthology.task.internal:Configuring sudo... 2024-02-20T12:39:15.217 DEBUG:teuthology.orchestra.run.smithi064:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-02-20T12:39:15.236 DEBUG:teuthology.orchestra.run.smithi071:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-02-20T12:39:15.280 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-02-20T12:39:15.290 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-02-20T12:39:15.291 DEBUG:teuthology.orchestra.run.smithi064:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-02-20T12:39:15.306 DEBUG:teuthology.orchestra.run.smithi071:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-02-20T12:39:15.339 DEBUG:teuthology.orchestra.run.smithi064:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-02-20T12:39:15.404 DEBUG:teuthology.orchestra.run.smithi064:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-02-20T12:39:15.605 DEBUG:teuthology.orchestra.run.smithi064:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-02-20T12:39:15.651 DEBUG:teuthology.orchestra.run.smithi064:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-02-20T12:39:15.691 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-02-20T12:39:15.691 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-02-20T12:39:15.759 DEBUG:teuthology.orchestra.run.smithi071:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-02-20T12:39:15.804 DEBUG:teuthology.orchestra.run.smithi071:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-02-20T12:39:15.857 DEBUG:teuthology.orchestra.run.smithi071:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-02-20T12:39:15.944 DEBUG:teuthology.orchestra.run.smithi071:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-02-20T12:39:15.987 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-02-20T12:39:15.987 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-02-20T12:39:16.054 DEBUG:teuthology.orchestra.run.smithi064:> sudo service rsyslog restart 2024-02-20T12:39:16.056 DEBUG:teuthology.orchestra.run.smithi071:> sudo service rsyslog restart 2024-02-20T12:39:16.110 INFO:teuthology.orchestra.run.smithi064.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-02-20T12:39:16.172 INFO:teuthology.orchestra.run.smithi071.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-02-20T12:39:16.560 INFO:teuthology.run_tasks:Running task internal.timer... 2024-02-20T12:39:16.569 INFO:teuthology.task.internal:Starting timer... 2024-02-20T12:39:16.569 INFO:teuthology.run_tasks:Running task pcp... 2024-02-20T12:39:16.579 INFO:teuthology.run_tasks:Running task selinux... 2024-02-20T12:39:16.586 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']} 2024-02-20T12:39:16.587 DEBUG:teuthology.orchestra.run.smithi064:> sudo service auditd rotate 2024-02-20T12:39:16.673 INFO:teuthology.orchestra.run.smithi064.stdout:Rotating logs: 2024-02-20T12:39:16.674 DEBUG:teuthology.orchestra.run.smithi071:> sudo service auditd rotate 2024-02-20T12:39:16.753 INFO:teuthology.orchestra.run.smithi071.stdout:Rotating logs: 2024-02-20T12:39:16.755 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-02-20T12:39:16.755 DEBUG:teuthology.orchestra.run.smithi064:> /usr/sbin/getenforce 2024-02-20T12:39:16.816 INFO:teuthology.orchestra.run.smithi064.stdout:Permissive 2024-02-20T12:39:16.817 DEBUG:teuthology.orchestra.run.smithi071:> /usr/sbin/getenforce 2024-02-20T12:39:16.839 INFO:teuthology.orchestra.run.smithi071.stdout:Permissive 2024-02-20T12:39:16.839 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi064.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi071.front.sepia.ceph.com': 'permissive'} 2024-02-20T12:39:16.839 DEBUG:teuthology.orchestra.run.smithi064:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-02-20T12:39:16.868 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-20T12:39:16.869 DEBUG:teuthology.orchestra.run.smithi071:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-02-20T12:39:16.907 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-20T12:39:16.908 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-02-20T12:39:16.908 DEBUG:teuthology.orchestra.run.smithi064:> sudo /usr/sbin/setenforce permissive 2024-02-20T12:39:16.936 DEBUG:teuthology.orchestra.run.smithi071:> sudo /usr/sbin/setenforce permissive 2024-02-20T12:39:16.974 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-02-20T12:39:16.985 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-02-20T12:39:16.989 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-02-20T12:39:17.096 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-02-20T12:39:17.114 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-02-20T12:39:17.116 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi064.front.sepia.ceph.com,smithi071.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-02-20T12:45:14.157 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi064.front.sepia.ceph.com'), Remote(name='ubuntu@smithi071.front.sepia.ceph.com')] 2024-02-20T12:45:14.182 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi064.front.sepia.ceph.com' 2024-02-20T12:45:14.184 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi064.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-20T12:45:14.265 DEBUG:teuthology.orchestra.run.smithi064:> true 2024-02-20T12:45:14.336 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi064.front.sepia.ceph.com' 2024-02-20T12:45:14.336 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi071.front.sepia.ceph.com' 2024-02-20T12:45:14.337 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi071.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-20T12:45:14.418 DEBUG:teuthology.orchestra.run.smithi071:> true 2024-02-20T12:45:14.493 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi071.front.sepia.ceph.com' 2024-02-20T12:45:14.493 INFO:teuthology.run_tasks:Running task clock... 2024-02-20T12:45:14.538 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-02-20T12:45:14.539 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-02-20T12:45:14.539 DEBUG:teuthology.orchestra.run.smithi064:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-02-20T12:45:14.548 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-02-20T12:45:14.548 DEBUG:teuthology.orchestra.run.smithi071:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-02-20T12:45:14.582 INFO:teuthology.orchestra.run.smithi064.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-02-20T12:45:14.588 INFO:teuthology.orchestra.run.smithi071.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-02-20T12:45:14.597 INFO:teuthology.orchestra.run.smithi064.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-02-20T12:45:14.603 INFO:teuthology.orchestra.run.smithi071.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-02-20T12:45:14.662 INFO:teuthology.orchestra.run.smithi064.stderr:sudo: ntpd: command not found 2024-02-20T12:45:14.672 INFO:teuthology.orchestra.run.smithi071.stderr:sudo: ntpd: command not found 2024-02-20T12:45:14.674 INFO:teuthology.orchestra.run.smithi064.stdout:506 Cannot talk to daemon 2024-02-20T12:45:14.684 INFO:teuthology.orchestra.run.smithi071.stdout:506 Cannot talk to daemon 2024-02-20T12:45:14.688 INFO:teuthology.orchestra.run.smithi064.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-02-20T12:45:14.697 INFO:teuthology.orchestra.run.smithi071.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-02-20T12:45:14.702 INFO:teuthology.orchestra.run.smithi064.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-02-20T12:45:14.711 INFO:teuthology.orchestra.run.smithi071.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-02-20T12:45:14.771 INFO:teuthology.orchestra.run.smithi064.stderr:bash: ntpq: command not found 2024-02-20T12:45:14.775 INFO:teuthology.orchestra.run.smithi064.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-02-20T12:45:14.775 INFO:teuthology.orchestra.run.smithi064.stdout:=============================================================================== 2024-02-20T12:45:14.776 INFO:teuthology.orchestra.run.smithi064.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-20T12:45:14.776 INFO:teuthology.orchestra.run.smithi064.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-20T12:45:14.776 INFO:teuthology.orchestra.run.smithi064.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-20T12:45:14.776 INFO:teuthology.orchestra.run.smithi064.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-20T12:45:14.776 INFO:teuthology.orchestra.run.smithi071.stderr:bash: ntpq: command not found 2024-02-20T12:45:14.779 INFO:teuthology.orchestra.run.smithi071.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-02-20T12:45:14.779 INFO:teuthology.orchestra.run.smithi071.stdout:=============================================================================== 2024-02-20T12:45:14.779 INFO:teuthology.orchestra.run.smithi071.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-20T12:45:14.779 INFO:teuthology.orchestra.run.smithi071.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-20T12:45:14.779 INFO:teuthology.orchestra.run.smithi071.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-20T12:45:14.780 INFO:teuthology.orchestra.run.smithi071.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-20T12:45:14.781 INFO:teuthology.run_tasks:Running task pexec... 2024-02-20T12:45:14.795 INFO:teuthology.task.pexec:Executing custom commands... 2024-02-20T12:45:14.795 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi064.front.sepia.ceph.com 2024-02-20T12:45:14.795 DEBUG:teuthology.orchestra.run.smithi064:> TESTDIR=/home/ubuntu/cephtest bash -s 2024-02-20T12:45:14.796 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi071.front.sepia.ceph.com 2024-02-20T12:45:14.797 DEBUG:teuthology.orchestra.run.smithi071:> TESTDIR=/home/ubuntu/cephtest bash -s 2024-02-20T12:45:15.606 INFO:teuthology.orchestra.run.smithi071.stdout:Last metadata expiration check: 0:01:33 ago on Tue 20 Feb 2024 12:43:42 PM UTC. 2024-02-20T12:45:15.643 INFO:teuthology.orchestra.run.smithi064.stdout:Last metadata expiration check: 0:01:24 ago on Tue 20 Feb 2024 12:43:51 PM UTC. 2024-02-20T12:45:17.621 INFO:teuthology.orchestra.run.smithi071.stdout:Dependencies resolved. 2024-02-20T12:45:17.622 INFO:teuthology.orchestra.run.smithi071.stdout:================================================================================ 2024-02-20T12:45:17.622 INFO:teuthology.orchestra.run.smithi071.stdout: Package Architecture Version Repository Size 2024-02-20T12:45:17.622 INFO:teuthology.orchestra.run.smithi071.stdout:================================================================================ 2024-02-20T12:45:17.622 INFO:teuthology.orchestra.run.smithi071.stdout:Resetting modules: 2024-02-20T12:45:17.622 INFO:teuthology.orchestra.run.smithi071.stdout: container-tools 2024-02-20T12:45:17.622 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-20T12:45:17.622 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction Summary 2024-02-20T12:45:17.622 INFO:teuthology.orchestra.run.smithi071.stdout:================================================================================ 2024-02-20T12:45:17.622 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-20T12:45:17.839 INFO:teuthology.orchestra.run.smithi064.stdout:Dependencies resolved. 2024-02-20T12:45:17.840 INFO:teuthology.orchestra.run.smithi064.stdout:================================================================================ 2024-02-20T12:45:17.840 INFO:teuthology.orchestra.run.smithi064.stdout: Package Architecture Version Repository Size 2024-02-20T12:45:17.840 INFO:teuthology.orchestra.run.smithi064.stdout:================================================================================ 2024-02-20T12:45:17.840 INFO:teuthology.orchestra.run.smithi064.stdout:Resetting modules: 2024-02-20T12:45:17.840 INFO:teuthology.orchestra.run.smithi064.stdout: container-tools 2024-02-20T12:45:17.840 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-20T12:45:17.840 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction Summary 2024-02-20T12:45:17.841 INFO:teuthology.orchestra.run.smithi064.stdout:================================================================================ 2024-02-20T12:45:17.841 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-20T12:45:17.879 INFO:teuthology.orchestra.run.smithi071.stdout:Complete! 2024-02-20T12:45:18.104 INFO:teuthology.orchestra.run.smithi064.stdout:Complete! 2024-02-20T12:45:18.736 INFO:teuthology.orchestra.run.smithi071.stdout:Last metadata expiration check: 0:01:36 ago on Tue 20 Feb 2024 12:43:42 PM UTC. 2024-02-20T12:45:18.974 INFO:teuthology.orchestra.run.smithi064.stdout:Last metadata expiration check: 0:01:27 ago on Tue 20 Feb 2024 12:43:51 PM UTC. 2024-02-20T12:45:20.770 INFO:teuthology.orchestra.run.smithi071.stdout:Dependencies resolved. 2024-02-20T12:45:20.806 INFO:teuthology.orchestra.run.smithi071.stdout:================================================================================ 2024-02-20T12:45:20.806 INFO:teuthology.orchestra.run.smithi071.stdout: Package Arch Version Repository Size 2024-02-20T12:45:20.806 INFO:teuthology.orchestra.run.smithi071.stdout:================================================================================ 2024-02-20T12:45:20.806 INFO:teuthology.orchestra.run.smithi071.stdout:Installing group/module packages: 2024-02-20T12:45:20.806 INFO:teuthology.orchestra.run.smithi071.stdout: aardvark-dns x86_64 2:1.10.0-1.module_el8+871+552be958 appstream 1.0 M 2024-02-20T12:45:20.806 INFO:teuthology.orchestra.run.smithi071.stdout: buildah x86_64 2:1.33.5-1.module_el8+885+7da147f3 appstream 9.7 M 2024-02-20T12:45:20.806 INFO:teuthology.orchestra.run.smithi071.stdout: cockpit-podman noarch 83-1.module_el8+847+7863d4e6 appstream 688 k 2024-02-20T12:45:20.807 INFO:teuthology.orchestra.run.smithi071.stdout: crun x86_64 1.14-1.module_el8+871+552be958 appstream 257 k 2024-02-20T12:45:20.807 INFO:teuthology.orchestra.run.smithi071.stdout: netavark x86_64 2:1.10.2-1.module_el8+885+7da147f3 appstream 4.3 M 2024-02-20T12:45:20.807 INFO:teuthology.orchestra.run.smithi071.stdout: python3-podman noarch 4.9.0-1.module_el8+847+7863d4e6 appstream 180 k 2024-02-20T12:45:20.807 INFO:teuthology.orchestra.run.smithi071.stdout: skopeo x86_64 2:1.14.3-0.1.module_el8+885+7da147f3 appstream 8.8 M 2024-02-20T12:45:20.807 INFO:teuthology.orchestra.run.smithi071.stdout: toolbox x86_64 0.0.99.5-1.module_el8+847+7863d4e6 appstream 2.5 M 2024-02-20T12:45:20.807 INFO:teuthology.orchestra.run.smithi071.stdout: udica noarch 0.2.6-20.module_el8+804+f131391c appstream 53 k 2024-02-20T12:45:20.807 INFO:teuthology.orchestra.run.smithi071.stdout:Installing dependencies: 2024-02-20T12:45:20.807 INFO:teuthology.orchestra.run.smithi071.stdout: python3-pytoml noarch 0.1.14-5.git7dea353.el8 appstream 25 k 2024-02-20T12:45:20.807 INFO:teuthology.orchestra.run.smithi071.stdout: python3-pyxdg noarch 0.25-16.el8 appstream 94 k 2024-02-20T12:45:20.807 INFO:teuthology.orchestra.run.smithi071.stdout: yajl x86_64 2.1.0-12.el8 appstream 41 k 2024-02-20T12:45:20.807 INFO:teuthology.orchestra.run.smithi071.stdout:Installing weak dependencies: 2024-02-20T12:45:20.807 INFO:teuthology.orchestra.run.smithi071.stdout: criu-libs x86_64 3.15-3.module_el8.7.0+1216+b022c01d appstream 38 k 2024-02-20T12:45:20.807 INFO:teuthology.orchestra.run.smithi071.stdout:Installing module profiles: 2024-02-20T12:45:20.807 INFO:teuthology.orchestra.run.smithi071.stdout: container-tools/common 2024-02-20T12:45:20.808 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-20T12:45:20.808 INFO:teuthology.orchestra.run.smithi071.stdout:Enabling module streams: 2024-02-20T12:45:20.808 INFO:teuthology.orchestra.run.smithi071.stdout: container-tools rhel8 2024-02-20T12:45:20.808 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-20T12:45:20.808 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction Summary 2024-02-20T12:45:20.808 INFO:teuthology.orchestra.run.smithi071.stdout:================================================================================ 2024-02-20T12:45:20.808 INFO:teuthology.orchestra.run.smithi071.stdout:Install 13 Packages 2024-02-20T12:45:20.808 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-20T12:45:20.809 INFO:teuthology.orchestra.run.smithi071.stdout:Total download size: 28 M 2024-02-20T12:45:20.809 INFO:teuthology.orchestra.run.smithi071.stdout:Installed size: 92 M 2024-02-20T12:45:20.809 INFO:teuthology.orchestra.run.smithi071.stdout:Downloading Packages: 2024-02-20T12:45:21.131 INFO:teuthology.orchestra.run.smithi064.stdout:Dependencies resolved. 2024-02-20T12:45:21.170 INFO:teuthology.orchestra.run.smithi064.stdout:================================================================================ 2024-02-20T12:45:21.170 INFO:teuthology.orchestra.run.smithi064.stdout: Package Arch Version Repository Size 2024-02-20T12:45:21.170 INFO:teuthology.orchestra.run.smithi064.stdout:================================================================================ 2024-02-20T12:45:21.170 INFO:teuthology.orchestra.run.smithi064.stdout:Installing group/module packages: 2024-02-20T12:45:21.170 INFO:teuthology.orchestra.run.smithi064.stdout: aardvark-dns x86_64 2:1.10.0-1.module_el8+871+552be958 appstream 1.0 M 2024-02-20T12:45:21.170 INFO:teuthology.orchestra.run.smithi064.stdout: buildah x86_64 2:1.33.5-1.module_el8+885+7da147f3 appstream 9.7 M 2024-02-20T12:45:21.170 INFO:teuthology.orchestra.run.smithi064.stdout: cockpit-podman noarch 83-1.module_el8+847+7863d4e6 appstream 688 k 2024-02-20T12:45:21.170 INFO:teuthology.orchestra.run.smithi064.stdout: crun x86_64 1.14-1.module_el8+871+552be958 appstream 257 k 2024-02-20T12:45:21.170 INFO:teuthology.orchestra.run.smithi064.stdout: netavark x86_64 2:1.10.2-1.module_el8+885+7da147f3 appstream 4.3 M 2024-02-20T12:45:21.170 INFO:teuthology.orchestra.run.smithi064.stdout: python3-podman noarch 4.9.0-1.module_el8+847+7863d4e6 appstream 180 k 2024-02-20T12:45:21.170 INFO:teuthology.orchestra.run.smithi064.stdout: skopeo x86_64 2:1.14.3-0.1.module_el8+885+7da147f3 appstream 8.8 M 2024-02-20T12:45:21.170 INFO:teuthology.orchestra.run.smithi064.stdout: toolbox x86_64 0.0.99.5-1.module_el8+847+7863d4e6 appstream 2.5 M 2024-02-20T12:45:21.171 INFO:teuthology.orchestra.run.smithi064.stdout: udica noarch 0.2.6-20.module_el8+804+f131391c appstream 53 k 2024-02-20T12:45:21.171 INFO:teuthology.orchestra.run.smithi064.stdout:Installing dependencies: 2024-02-20T12:45:21.171 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pytoml noarch 0.1.14-5.git7dea353.el8 appstream 25 k 2024-02-20T12:45:21.171 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pyxdg noarch 0.25-16.el8 appstream 94 k 2024-02-20T12:45:21.171 INFO:teuthology.orchestra.run.smithi064.stdout: yajl x86_64 2.1.0-12.el8 appstream 41 k 2024-02-20T12:45:21.171 INFO:teuthology.orchestra.run.smithi064.stdout:Installing weak dependencies: 2024-02-20T12:45:21.171 INFO:teuthology.orchestra.run.smithi064.stdout: criu-libs x86_64 3.15-3.module_el8.7.0+1216+b022c01d appstream 38 k 2024-02-20T12:45:21.171 INFO:teuthology.orchestra.run.smithi064.stdout:Installing module profiles: 2024-02-20T12:45:21.171 INFO:teuthology.orchestra.run.smithi064.stdout: container-tools/common 2024-02-20T12:45:21.171 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-20T12:45:21.171 INFO:teuthology.orchestra.run.smithi064.stdout:Enabling module streams: 2024-02-20T12:45:21.171 INFO:teuthology.orchestra.run.smithi064.stdout: container-tools rhel8 2024-02-20T12:45:21.172 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-20T12:45:21.172 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction Summary 2024-02-20T12:45:21.172 INFO:teuthology.orchestra.run.smithi064.stdout:================================================================================ 2024-02-20T12:45:21.172 INFO:teuthology.orchestra.run.smithi064.stdout:Install 13 Packages 2024-02-20T12:45:21.172 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-20T12:45:21.172 INFO:teuthology.orchestra.run.smithi064.stdout:Total download size: 28 M 2024-02-20T12:45:21.173 INFO:teuthology.orchestra.run.smithi064.stdout:Installed size: 92 M 2024-02-20T12:45:21.173 INFO:teuthology.orchestra.run.smithi064.stdout:Downloading Packages: 2024-02-20T12:45:21.559 INFO:teuthology.orchestra.run.smithi071.stdout:(1/13): cockpit-podman-83-1.module_el8+847+7863 1.2 MB/s | 688 kB 00:00 2024-02-20T12:45:21.617 INFO:teuthology.orchestra.run.smithi071.stdout:(2/13): aardvark-dns-1.10.0-1.module_el8+871+55 1.6 MB/s | 1.0 MB 00:00 2024-02-20T12:45:21.676 INFO:teuthology.orchestra.run.smithi071.stdout:(3/13): criu-libs-3.15-3.module_el8.7.0+1216+b0 324 kB/s | 38 kB 00:00 2024-02-20T12:45:21.760 INFO:teuthology.orchestra.run.smithi071.stdout:(4/13): crun-1.14-1.module_el8+871+552be958.x86 1.8 MB/s | 257 kB 00:00 2024-02-20T12:45:21.820 INFO:teuthology.orchestra.run.smithi064.stdout:(1/13): cockpit-podman-83-1.module_el8+847+7863 1.1 MB/s | 688 kB 00:00 2024-02-20T12:45:21.853 INFO:teuthology.orchestra.run.smithi064.stdout:(2/13): aardvark-dns-1.10.0-1.module_el8+871+55 1.6 MB/s | 1.0 MB 00:00 2024-02-20T12:45:21.860 INFO:teuthology.orchestra.run.smithi071.stdout:(5/13): python3-podman-4.9.0-1.module_el8+847+7 1.8 MB/s | 180 kB 00:00 2024-02-20T12:45:21.929 INFO:teuthology.orchestra.run.smithi064.stdout:(3/13): criu-libs-3.15-3.module_el8.7.0+1216+b0 349 kB/s | 38 kB 00:00 2024-02-20T12:45:21.954 INFO:teuthology.orchestra.run.smithi064.stdout:(4/13): crun-1.14-1.module_el8+871+552be958.x86 2.5 MB/s | 257 kB 00:00 2024-02-20T12:45:22.063 INFO:teuthology.orchestra.run.smithi064.stdout:(5/13): python3-podman-4.9.0-1.module_el8+847+7 1.6 MB/s | 180 kB 00:00 2024-02-20T12:45:22.068 INFO:teuthology.orchestra.run.smithi071.stdout:(6/13): buildah-1.33.5-1.module_el8+885+7da147f 9.0 MB/s | 9.7 MB 00:01 2024-02-20T12:45:22.085 INFO:teuthology.orchestra.run.smithi071.stdout:(7/13): python3-pytoml-0.1.14-5.git7dea353.el8. 110 kB/s | 25 kB 00:00 2024-02-20T12:45:22.163 INFO:teuthology.orchestra.run.smithi064.stdout:(6/13): python3-pytoml-0.1.14-5.git7dea353.el8. 248 kB/s | 25 kB 00:00 2024-02-20T12:45:22.169 INFO:teuthology.orchestra.run.smithi071.stdout:(8/13): python3-pyxdg-0.25-16.el8.noarch.rpm 938 kB/s | 94 kB 00:00 2024-02-20T12:45:22.271 INFO:teuthology.orchestra.run.smithi064.stdout:(7/13): python3-pyxdg-0.25-16.el8.noarch.rpm 865 kB/s | 94 kB 00:00 2024-02-20T12:45:22.344 INFO:teuthology.orchestra.run.smithi071.stdout:(9/13): netavark-1.10.2-1.module_el8+885+7da147 6.5 MB/s | 4.3 MB 00:00 2024-02-20T12:45:22.449 INFO:teuthology.orchestra.run.smithi071.stdout:(10/13): toolbox-0.0.99.5-1.module_el8+847+7863 9.0 MB/s | 2.5 MB 00:00 2024-02-20T12:45:22.470 INFO:teuthology.orchestra.run.smithi071.stdout:(11/13): udica-0.2.6-20.module_el8+804+f131391c 420 kB/s | 53 kB 00:00 2024-02-20T12:45:22.522 INFO:teuthology.orchestra.run.smithi064.stdout:(8/13): netavark-1.10.2-1.module_el8+885+7da147 7.3 MB/s | 4.3 MB 00:00 2024-02-20T12:45:22.536 INFO:teuthology.orchestra.run.smithi071.stdout:(12/13): yajl-2.1.0-12.el8.x86_64.rpm 469 kB/s | 41 kB 00:00 2024-02-20T12:45:22.860 INFO:teuthology.orchestra.run.smithi071.stdout:(13/13): skopeo-1.14.3-0.1.module_el8+885+7da14 11 MB/s | 8.8 MB 00:00 2024-02-20T12:45:22.861 INFO:teuthology.orchestra.run.smithi071.stdout:-------------------------------------------------------------------------------- 2024-02-20T12:45:22.861 INFO:teuthology.orchestra.run.smithi071.stdout:Total 13 MB/s | 28 MB 00:02 2024-02-20T12:45:22.864 INFO:teuthology.orchestra.run.smithi064.stdout:(9/13): toolbox-0.0.99.5-1.module_el8+847+7863d 7.4 MB/s | 2.5 MB 00:00 2024-02-20T12:45:22.964 INFO:teuthology.orchestra.run.smithi064.stdout:(10/13): skopeo-1.14.3-0.1.module_el8+885+7da14 13 MB/s | 8.8 MB 00:00 2024-02-20T12:45:22.989 INFO:teuthology.orchestra.run.smithi064.stdout:(11/13): udica-0.2.6-20.module_el8+804+f131391c 420 kB/s | 53 kB 00:00 2024-02-20T12:45:23.123 INFO:teuthology.orchestra.run.smithi064.stdout:(12/13): buildah-1.33.5-1.module_el8+885+7da147 5.1 MB/s | 9.7 MB 00:01 2024-02-20T12:45:23.130 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction check 2024-02-20T12:45:23.148 INFO:teuthology.orchestra.run.smithi064.stdout:(13/13): yajl-2.1.0-12.el8.x86_64.rpm 223 kB/s | 41 kB 00:00 2024-02-20T12:45:23.148 INFO:teuthology.orchestra.run.smithi064.stdout:-------------------------------------------------------------------------------- 2024-02-20T12:45:23.148 INFO:teuthology.orchestra.run.smithi064.stdout:Total 14 MB/s | 28 MB 00:01 2024-02-20T12:45:23.150 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction check succeeded. 2024-02-20T12:45:23.150 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction test 2024-02-20T12:45:23.416 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction test succeeded. 2024-02-20T12:45:23.419 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction 2024-02-20T12:45:23.424 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction check 2024-02-20T12:45:23.447 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction check succeeded. 2024-02-20T12:45:23.447 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction test 2024-02-20T12:45:23.731 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction test succeeded. 2024-02-20T12:45:23.734 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction 2024-02-20T12:45:24.137 INFO:teuthology.orchestra.run.smithi071.stdout: Preparing : 1/1 2024-02-20T12:45:24.509 INFO:teuthology.orchestra.run.smithi064.stdout: Preparing : 1/1 2024-02-20T12:45:25.100 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : yajl-2.1.0-12.el8.x86_64 1/13 2024-02-20T12:45:25.335 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : skopeo-2:1.14.3-0.1.module_el8+885+7da147f3.x86_64 2/13 2024-02-20T12:45:25.488 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-pyxdg-0.25-16.el8.noarch 3/13 2024-02-20T12:45:25.575 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : yajl-2.1.0-12.el8.x86_64 1/13 2024-02-20T12:45:25.708 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 4/13 2024-02-20T12:45:25.817 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : skopeo-2:1.14.3-0.1.module_el8+885+7da147f3.x86_64 2/13 2024-02-20T12:45:25.915 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : criu-libs-3.15-3.module_el8.7.0+1216+b022c01d.x86_ 5/13 2024-02-20T12:45:26.019 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-pyxdg-0.25-16.el8.noarch 3/13 2024-02-20T12:45:26.247 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 4/13 2024-02-20T12:45:26.469 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : criu-libs-3.15-3.module_el8.7.0+1216+b022c01d.x86_ 5/13 2024-02-20T12:45:26.510 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : aardvark-dns-2:1.10.0-1.module_el8+871+552be958.x8 6/13 2024-02-20T12:45:27.217 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : aardvark-dns-2:1.10.0-1.module_el8+871+552be958.x8 6/13 2024-02-20T12:45:27.466 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : netavark-2:1.10.2-1.module_el8+885+7da147f3.x86_64 7/13 2024-02-20T12:45:27.707 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : buildah-2:1.33.5-1.module_el8+885+7da147f3.x86_64 8/13 2024-02-20T12:45:27.951 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : crun-1.14-1.module_el8+871+552be958.x86_64 9/13 2024-02-20T12:45:28.263 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : netavark-2:1.10.2-1.module_el8+885+7da147f3.x86_64 7/13 2024-02-20T12:45:28.439 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-podman-4.9.0-1.module_el8+847+7863d4e6.noa 10/13 2024-02-20T12:45:28.658 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : buildah-2:1.33.5-1.module_el8+885+7da147f3.x86_64 8/13 2024-02-20T12:45:28.672 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : toolbox-0.0.99.5-1.module_el8+847+7863d4e6.x86_64 11/13 2024-02-20T12:45:28.916 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : crun-1.14-1.module_el8+871+552be958.x86_64 9/13 2024-02-20T12:45:29.319 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : udica-0.2.6-20.module_el8+804+f131391c.noarch 12/13 2024-02-20T12:45:29.567 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-podman-4.9.0-1.module_el8+847+7863d4e6.noa 10/13 2024-02-20T12:45:29.596 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : cockpit-podman-83-1.module_el8+847+7863d4e6.noarch 13/13 2024-02-20T12:45:29.957 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : toolbox-0.0.99.5-1.module_el8+847+7863d4e6.x86_64 11/13 2024-02-20T12:45:30.027 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: cockpit-podman-83-1.module_el8+847+7863d4e6.noarch 13/13 2024-02-20T12:45:30.027 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : aardvark-dns-2:1.10.0-1.module_el8+871+552be958.x8 1/13 2024-02-20T12:45:30.027 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : buildah-2:1.33.5-1.module_el8+885+7da147f3.x86_64 2/13 2024-02-20T12:45:30.027 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : cockpit-podman-83-1.module_el8+847+7863d4e6.noarch 3/13 2024-02-20T12:45:30.028 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : criu-libs-3.15-3.module_el8.7.0+1216+b022c01d.x86_ 4/13 2024-02-20T12:45:30.028 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : crun-1.14-1.module_el8+871+552be958.x86_64 5/13 2024-02-20T12:45:30.028 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : netavark-2:1.10.2-1.module_el8+885+7da147f3.x86_64 6/13 2024-02-20T12:45:30.028 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-podman-4.9.0-1.module_el8+847+7863d4e6.noa 7/13 2024-02-20T12:45:30.028 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 8/13 2024-02-20T12:45:30.028 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-pyxdg-0.25-16.el8.noarch 9/13 2024-02-20T12:45:30.028 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : skopeo-2:1.14.3-0.1.module_el8+885+7da147f3.x86_64 10/13 2024-02-20T12:45:30.028 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : toolbox-0.0.99.5-1.module_el8+847+7863d4e6.x86_64 11/13 2024-02-20T12:45:30.028 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : udica-0.2.6-20.module_el8+804+f131391c.noarch 12/13 2024-02-20T12:45:30.204 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : udica-0.2.6-20.module_el8+804+f131391c.noarch 12/13 2024-02-20T12:45:30.435 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : cockpit-podman-83-1.module_el8+847+7863d4e6.noarch 13/13 2024-02-20T12:45:30.521 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : yajl-2.1.0-12.el8.x86_64 13/13 2024-02-20T12:45:30.521 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-20T12:45:30.521 INFO:teuthology.orchestra.run.smithi071.stdout:Installed: 2024-02-20T12:45:30.521 INFO:teuthology.orchestra.run.smithi071.stdout: aardvark-dns-2:1.10.0-1.module_el8+871+552be958.x86_64 2024-02-20T12:45:30.521 INFO:teuthology.orchestra.run.smithi071.stdout: buildah-2:1.33.5-1.module_el8+885+7da147f3.x86_64 2024-02-20T12:45:30.521 INFO:teuthology.orchestra.run.smithi071.stdout: cockpit-podman-83-1.module_el8+847+7863d4e6.noarch 2024-02-20T12:45:30.522 INFO:teuthology.orchestra.run.smithi071.stdout: criu-libs-3.15-3.module_el8.7.0+1216+b022c01d.x86_64 2024-02-20T12:45:30.522 INFO:teuthology.orchestra.run.smithi071.stdout: crun-1.14-1.module_el8+871+552be958.x86_64 2024-02-20T12:45:30.522 INFO:teuthology.orchestra.run.smithi071.stdout: netavark-2:1.10.2-1.module_el8+885+7da147f3.x86_64 2024-02-20T12:45:30.522 INFO:teuthology.orchestra.run.smithi071.stdout: python3-podman-4.9.0-1.module_el8+847+7863d4e6.noarch 2024-02-20T12:45:30.522 INFO:teuthology.orchestra.run.smithi071.stdout: python3-pytoml-0.1.14-5.git7dea353.el8.noarch 2024-02-20T12:45:30.522 INFO:teuthology.orchestra.run.smithi071.stdout: python3-pyxdg-0.25-16.el8.noarch 2024-02-20T12:45:30.522 INFO:teuthology.orchestra.run.smithi071.stdout: skopeo-2:1.14.3-0.1.module_el8+885+7da147f3.x86_64 2024-02-20T12:45:30.522 INFO:teuthology.orchestra.run.smithi071.stdout: toolbox-0.0.99.5-1.module_el8+847+7863d4e6.x86_64 2024-02-20T12:45:30.522 INFO:teuthology.orchestra.run.smithi071.stdout: udica-0.2.6-20.module_el8+804+f131391c.noarch 2024-02-20T12:45:30.522 INFO:teuthology.orchestra.run.smithi071.stdout: yajl-2.1.0-12.el8.x86_64 2024-02-20T12:45:30.522 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-20T12:45:30.522 INFO:teuthology.orchestra.run.smithi071.stdout:Complete! 2024-02-20T12:45:30.801 DEBUG:teuthology.parallel:result is None 2024-02-20T12:45:30.887 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: cockpit-podman-83-1.module_el8+847+7863d4e6.noarch 13/13 2024-02-20T12:45:30.887 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : aardvark-dns-2:1.10.0-1.module_el8+871+552be958.x8 1/13 2024-02-20T12:45:30.887 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : buildah-2:1.33.5-1.module_el8+885+7da147f3.x86_64 2/13 2024-02-20T12:45:30.887 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : cockpit-podman-83-1.module_el8+847+7863d4e6.noarch 3/13 2024-02-20T12:45:30.887 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : criu-libs-3.15-3.module_el8.7.0+1216+b022c01d.x86_ 4/13 2024-02-20T12:45:30.887 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : crun-1.14-1.module_el8+871+552be958.x86_64 5/13 2024-02-20T12:45:30.887 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : netavark-2:1.10.2-1.module_el8+885+7da147f3.x86_64 6/13 2024-02-20T12:45:30.887 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-podman-4.9.0-1.module_el8+847+7863d4e6.noa 7/13 2024-02-20T12:45:30.887 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 8/13 2024-02-20T12:45:30.887 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-pyxdg-0.25-16.el8.noarch 9/13 2024-02-20T12:45:30.887 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : skopeo-2:1.14.3-0.1.module_el8+885+7da147f3.x86_64 10/13 2024-02-20T12:45:30.888 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : toolbox-0.0.99.5-1.module_el8+847+7863d4e6.x86_64 11/13 2024-02-20T12:45:30.888 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : udica-0.2.6-20.module_el8+804+f131391c.noarch 12/13 2024-02-20T12:45:31.462 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : yajl-2.1.0-12.el8.x86_64 13/13 2024-02-20T12:45:31.463 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-20T12:45:31.463 INFO:teuthology.orchestra.run.smithi064.stdout:Installed: 2024-02-20T12:45:31.463 INFO:teuthology.orchestra.run.smithi064.stdout: aardvark-dns-2:1.10.0-1.module_el8+871+552be958.x86_64 2024-02-20T12:45:31.463 INFO:teuthology.orchestra.run.smithi064.stdout: buildah-2:1.33.5-1.module_el8+885+7da147f3.x86_64 2024-02-20T12:45:31.463 INFO:teuthology.orchestra.run.smithi064.stdout: cockpit-podman-83-1.module_el8+847+7863d4e6.noarch 2024-02-20T12:45:31.463 INFO:teuthology.orchestra.run.smithi064.stdout: criu-libs-3.15-3.module_el8.7.0+1216+b022c01d.x86_64 2024-02-20T12:45:31.463 INFO:teuthology.orchestra.run.smithi064.stdout: crun-1.14-1.module_el8+871+552be958.x86_64 2024-02-20T12:45:31.463 INFO:teuthology.orchestra.run.smithi064.stdout: netavark-2:1.10.2-1.module_el8+885+7da147f3.x86_64 2024-02-20T12:45:31.464 INFO:teuthology.orchestra.run.smithi064.stdout: python3-podman-4.9.0-1.module_el8+847+7863d4e6.noarch 2024-02-20T12:45:31.464 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pytoml-0.1.14-5.git7dea353.el8.noarch 2024-02-20T12:45:31.464 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pyxdg-0.25-16.el8.noarch 2024-02-20T12:45:31.464 INFO:teuthology.orchestra.run.smithi064.stdout: skopeo-2:1.14.3-0.1.module_el8+885+7da147f3.x86_64 2024-02-20T12:45:31.464 INFO:teuthology.orchestra.run.smithi064.stdout: toolbox-0.0.99.5-1.module_el8+847+7863d4e6.x86_64 2024-02-20T12:45:31.464 INFO:teuthology.orchestra.run.smithi064.stdout: udica-0.2.6-20.module_el8+804+f131391c.noarch 2024-02-20T12:45:31.464 INFO:teuthology.orchestra.run.smithi064.stdout: yajl-2.1.0-12.el8.x86_64 2024-02-20T12:45:31.464 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-20T12:45:31.464 INFO:teuthology.orchestra.run.smithi064.stdout:Complete! 2024-02-20T12:45:31.742 DEBUG:teuthology.parallel:result is None 2024-02-20T12:45:31.742 INFO:teuthology.run_tasks:Running task install... 2024-02-20T12:45:31.751 DEBUG:teuthology.task.install:project ceph 2024-02-20T12:45:31.751 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'b0bd01b8442d66b8a31f8cb3050334b026de2f3b'}} 2024-02-20T12:45:31.752 DEBUG:teuthology.task.install:config {'branch': 'pacific', 'exclude_packages': ['ceph-volume'], 'flavor': 'default', 'sha1': 'b0bd01b8442d66b8a31f8cb3050334b026de2f3b'} 2024-02-20T12:45:31.752 INFO:teuthology.task.install:Using flavor: default 2024-02-20T12:45:31.760 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-02-20T12:45:31.760 INFO:teuthology.task.install:extra packages: [] 2024-02-20T12:45:31.760 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': 'pacific', 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': ['ceph-volume'], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'b0bd01b8442d66b8a31f8cb3050334b026de2f3b', 'tag': None, 'wait_for_package': False} 2024-02-20T12:45:31.761 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-02-20T12:45:31.761 INFO:teuthology.packaging:ref: None 2024-02-20T12:45:31.761 INFO:teuthology.packaging:tag: None 2024-02-20T12:45:31.761 INFO:teuthology.packaging:branch: pacific 2024-02-20T12:45:31.761 INFO:teuthology.packaging:sha1: b0bd01b8442d66b8a31f8cb3050334b026de2f3b 2024-02-20T12:45:31.761 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=pacific 2024-02-20T12:45:31.763 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': 'pacific', 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': ['ceph-volume'], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'b0bd01b8442d66b8a31f8cb3050334b026de2f3b', 'tag': None, 'wait_for_package': False} 2024-02-20T12:45:31.763 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-02-20T12:45:31.763 INFO:teuthology.packaging:ref: None 2024-02-20T12:45:31.764 INFO:teuthology.packaging:tag: None 2024-02-20T12:45:31.764 INFO:teuthology.packaging:branch: pacific 2024-02-20T12:45:31.764 INFO:teuthology.packaging:sha1: b0bd01b8442d66b8a31f8cb3050334b026de2f3b 2024-02-20T12:45:31.764 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=pacific 2024-02-20T12:45:32.003 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/pacific/81bd20d634209c7cb82c18be12b4b5a05643ebf1/centos/8/flavors/default/ 2024-02-20T12:45:32.004 INFO:teuthology.task.install.rpm:Package version is 16.2.14-548.g81bd20d6 2024-02-20T12:45:32.120 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/pacific/81bd20d634209c7cb82c18be12b4b5a05643ebf1/centos/8/flavors/default/ 2024-02-20T12:45:32.120 INFO:teuthology.task.install.rpm:Package version is 16.2.14-548.g81bd20d6 2024-02-20T12:45:32.139 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/pacific/81bd20d634209c7cb82c18be12b4b5a05643ebf1/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/pacific/81bd20d634209c7cb82c18be12b4b5a05643ebf1/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/pacific/81bd20d634209c7cb82c18be12b4b5a05643ebf1/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-02-20T12:45:32.140 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-02-20T12:45:32.140 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-02-20T12:45:32.169 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-02-20T12:45:32.169 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-02-20T12:45:32.169 INFO:teuthology.packaging:ref: None 2024-02-20T12:45:32.169 INFO:teuthology.packaging:tag: None 2024-02-20T12:45:32.169 INFO:teuthology.packaging:branch: pacific 2024-02-20T12:45:32.169 INFO:teuthology.packaging:sha1: b0bd01b8442d66b8a31f8cb3050334b026de2f3b 2024-02-20T12:45:32.170 DEBUG:teuthology.orchestra.run.smithi064:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;ref/pacific/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-02-20T12:45:32.238 DEBUG:teuthology.orchestra.run.smithi064:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-02-20T12:45:32.314 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/pacific/81bd20d634209c7cb82c18be12b4b5a05643ebf1/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/pacific/81bd20d634209c7cb82c18be12b4b5a05643ebf1/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/pacific/81bd20d634209c7cb82c18be12b4b5a05643ebf1/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-02-20T12:45:32.314 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-02-20T12:45:32.314 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-02-20T12:45:32.316 DEBUG:teuthology.orchestra.run.smithi064:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-02-20T12:45:32.348 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-02-20T12:45:32.349 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-02-20T12:45:32.349 INFO:teuthology.packaging:ref: None 2024-02-20T12:45:32.349 INFO:teuthology.packaging:tag: None 2024-02-20T12:45:32.349 INFO:teuthology.packaging:branch: pacific 2024-02-20T12:45:32.349 INFO:teuthology.packaging:sha1: b0bd01b8442d66b8a31f8cb3050334b026de2f3b 2024-02-20T12:45:32.349 DEBUG:teuthology.orchestra.run.smithi071:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;ref/pacific/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-02-20T12:45:32.382 INFO:teuthology.orchestra.run.smithi064.stdout:check_obsoletes = 1 2024-02-20T12:45:32.384 DEBUG:teuthology.orchestra.run.smithi064:> sudo yum clean all 2024-02-20T12:45:32.421 DEBUG:teuthology.orchestra.run.smithi071:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-02-20T12:45:32.501 DEBUG:teuthology.orchestra.run.smithi071:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-02-20T12:45:32.568 INFO:teuthology.orchestra.run.smithi071.stdout:check_obsoletes = 1 2024-02-20T12:45:32.570 DEBUG:teuthology.orchestra.run.smithi071:> sudo yum clean all 2024-02-20T12:45:32.663 INFO:teuthology.orchestra.run.smithi064.stdout:76 files removed 2024-02-20T12:45:32.691 DEBUG:teuthology.orchestra.run.smithi064:> sudo yum -y install ceph-radosgw 2024-02-20T12:45:32.832 INFO:teuthology.orchestra.run.smithi071.stdout:76 files removed 2024-02-20T12:45:32.859 DEBUG:teuthology.orchestra.run.smithi071:> sudo yum -y install ceph-radosgw 2024-02-20T12:45:33.423 INFO:teuthology.orchestra.run.smithi064.stdout:ceph packages for x86_64 251 kB/s | 76 kB 00:00 2024-02-20T12:45:33.611 INFO:teuthology.orchestra.run.smithi071.stdout:ceph packages for x86_64 219 kB/s | 76 kB 00:00 2024-02-20T12:45:33.866 INFO:teuthology.orchestra.run.smithi064.stdout:ceph noarch packages 38 kB/s | 15 kB 00:00 2024-02-20T12:45:34.062 INFO:teuthology.orchestra.run.smithi071.stdout:ceph noarch packages 37 kB/s | 15 kB 00:00 2024-02-20T12:45:34.329 INFO:teuthology.orchestra.run.smithi071.stdout:ceph source packages 6.1 kB/s | 1.4 kB 00:00 2024-02-20T12:45:34.599 INFO:teuthology.orchestra.run.smithi064.stdout:ceph source packages 2.0 kB/s | 1.4 kB 00:00 2024-02-20T12:45:36.263 INFO:teuthology.orchestra.run.smithi071.stdout:CentOS Stream 8 - AppStream 14 MB/s | 27 MB 00:01 2024-02-20T12:45:39.384 INFO:teuthology.orchestra.run.smithi064.stdout:CentOS Stream 8 - AppStream 5.7 MB/s | 27 MB 00:04 2024-02-20T12:45:41.488 INFO:teuthology.orchestra.run.smithi071.stdout:CentOS Stream 8 - BaseOS 25 MB/s | 10 MB 00:00 2024-02-20T12:45:43.522 INFO:teuthology.orchestra.run.smithi071.stdout:CentOS Stream 8 - Extras 40 kB/s | 18 kB 00:00 2024-02-20T12:45:44.198 INFO:teuthology.orchestra.run.smithi071.stdout:CentOS Stream 8 - Extras common packages 12 kB/s | 6.9 kB 00:00 2024-02-20T12:45:44.832 INFO:teuthology.orchestra.run.smithi071.stdout:CentOS Stream 8 - PowerTools 8.9 MB/s | 4.8 MB 00:00 2024-02-20T12:45:45.026 INFO:teuthology.orchestra.run.smithi064.stdout:CentOS Stream 8 - BaseOS 20 MB/s | 10 MB 00:00 2024-02-20T12:45:45.925 INFO:teuthology.orchestra.run.smithi071.stdout:Copr repo for python3-asyncssh owned by ceph 20 kB/s | 3.5 kB 00:00 2024-02-20T12:45:46.994 INFO:teuthology.orchestra.run.smithi064.stdout:CentOS Stream 8 - Extras 63 kB/s | 18 kB 00:00 2024-02-20T12:45:47.303 INFO:teuthology.orchestra.run.smithi071.stdout:Extra Packages for Enterprise Linux 13 MB/s | 16 MB 00:01 2024-02-20T12:45:47.536 INFO:teuthology.orchestra.run.smithi064.stdout:CentOS Stream 8 - Extras common packages 16 kB/s | 6.9 kB 00:00 2024-02-20T12:45:48.237 INFO:teuthology.orchestra.run.smithi064.stdout:CentOS Stream 8 - PowerTools 8.1 MB/s | 4.8 MB 00:00 2024-02-20T12:45:49.338 INFO:teuthology.orchestra.run.smithi064.stdout:Copr repo for python3-asyncssh owned by ceph 29 kB/s | 3.5 kB 00:00 2024-02-20T12:45:50.317 INFO:teuthology.orchestra.run.smithi064.stdout:Extra Packages for Enterprise Linux 19 MB/s | 16 MB 00:00 2024-02-20T12:45:51.151 INFO:teuthology.orchestra.run.smithi071.stdout:lab-extras 255 kB/s | 24 kB 00:00 2024-02-20T12:45:54.339 INFO:teuthology.orchestra.run.smithi064.stdout:lab-extras 322 kB/s | 24 kB 00:00 2024-02-20T12:45:54.690 INFO:teuthology.orchestra.run.smithi071.stdout:Dependencies resolved. 2024-02-20T12:45:54.692 INFO:teuthology.orchestra.run.smithi071.stdout:================================================================================ 2024-02-20T12:45:54.692 INFO:teuthology.orchestra.run.smithi071.stdout: Package Arch Version Repository Size 2024-02-20T12:45:54.692 INFO:teuthology.orchestra.run.smithi071.stdout:================================================================================ 2024-02-20T12:45:54.692 INFO:teuthology.orchestra.run.smithi071.stdout:Installing: 2024-02-20T12:45:54.693 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-radosgw x86_64 2:16.2.14-548.g81bd20d6.el8 ceph 12 M 2024-02-20T12:45:54.693 INFO:teuthology.orchestra.run.smithi071.stdout:Upgrading: 2024-02-20T12:45:54.693 INFO:teuthology.orchestra.run.smithi071.stdout: librados2 x86_64 2:16.2.14-548.g81bd20d6.el8 ceph 3.7 M 2024-02-20T12:45:54.693 INFO:teuthology.orchestra.run.smithi071.stdout: librbd1 x86_64 2:16.2.14-548.g81bd20d6.el8 ceph 4.2 M 2024-02-20T12:45:54.693 INFO:teuthology.orchestra.run.smithi071.stdout:Installing dependencies: 2024-02-20T12:45:54.693 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-base x86_64 2:16.2.14-548.g81bd20d6.el8 ceph 6.1 M 2024-02-20T12:45:54.693 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-common x86_64 2:16.2.14-548.g81bd20d6.el8 ceph 24 M 2024-02-20T12:45:54.693 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-selinux x86_64 2:16.2.14-548.g81bd20d6.el8 ceph 26 k 2024-02-20T12:45:54.693 INFO:teuthology.orchestra.run.smithi071.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2024-02-20T12:45:54.693 INFO:teuthology.orchestra.run.smithi071.stdout: leveldb x86_64 1.22-1.el8 epel 181 k 2024-02-20T12:45:54.693 INFO:teuthology.orchestra.run.smithi071.stdout: libcephfs2 x86_64 2:16.2.14-548.g81bd20d6.el8 ceph 825 k 2024-02-20T12:45:54.693 INFO:teuthology.orchestra.run.smithi071.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2024-02-20T12:45:54.693 INFO:teuthology.orchestra.run.smithi071.stdout: librabbitmq x86_64 0.9.0-5.el8 baseos 47 k 2024-02-20T12:45:54.694 INFO:teuthology.orchestra.run.smithi071.stdout: libradosstriper1 x86_64 2:16.2.14-548.g81bd20d6.el8 ceph 494 k 2024-02-20T12:45:54.694 INFO:teuthology.orchestra.run.smithi071.stdout: librdkafka x86_64 1.6.1-1.el8 appstream 677 k 2024-02-20T12:45:54.694 INFO:teuthology.orchestra.run.smithi071.stdout: librgw2 x86_64 2:16.2.14-548.g81bd20d6.el8 ceph 3.8 M 2024-02-20T12:45:54.694 INFO:teuthology.orchestra.run.smithi071.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2024-02-20T12:45:54.694 INFO:teuthology.orchestra.run.smithi071.stdout: lttng-ust x86_64 2.8.1-11.el8 appstream 259 k 2024-02-20T12:45:54.694 INFO:teuthology.orchestra.run.smithi071.stdout: python3-ceph-argparse x86_64 2:16.2.14-548.g81bd20d6.el8 ceph 45 k 2024-02-20T12:45:54.694 INFO:teuthology.orchestra.run.smithi071.stdout: python3-ceph-common x86_64 2:16.2.14-548.g81bd20d6.el8 ceph 99 k 2024-02-20T12:45:54.694 INFO:teuthology.orchestra.run.smithi071.stdout: python3-cephfs x86_64 2:16.2.14-548.g81bd20d6.el8 ceph 215 k 2024-02-20T12:45:54.694 INFO:teuthology.orchestra.run.smithi071.stdout: python3-prettytable noarch 0.7.2-14.el8 appstream 44 k 2024-02-20T12:45:54.694 INFO:teuthology.orchestra.run.smithi071.stdout: python3-rados x86_64 2:16.2.14-548.g81bd20d6.el8 ceph 389 k 2024-02-20T12:45:54.694 INFO:teuthology.orchestra.run.smithi071.stdout: python3-rbd x86_64 2:16.2.14-548.g81bd20d6.el8 ceph 369 k 2024-02-20T12:45:54.694 INFO:teuthology.orchestra.run.smithi071.stdout: python3-rgw x86_64 2:16.2.14-548.g81bd20d6.el8 ceph 114 k 2024-02-20T12:45:54.694 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-20T12:45:54.694 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction Summary 2024-02-20T12:45:54.694 INFO:teuthology.orchestra.run.smithi071.stdout:================================================================================ 2024-02-20T12:45:54.695 INFO:teuthology.orchestra.run.smithi071.stdout:Install 21 Packages 2024-02-20T12:45:54.695 INFO:teuthology.orchestra.run.smithi071.stdout:Upgrade 2 Packages 2024-02-20T12:45:54.695 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-20T12:45:54.695 INFO:teuthology.orchestra.run.smithi071.stdout:Total download size: 58 M 2024-02-20T12:45:54.695 INFO:teuthology.orchestra.run.smithi071.stdout:Downloading Packages: 2024-02-20T12:45:56.385 INFO:teuthology.orchestra.run.smithi071.stdout:(1/23): ceph-base-16.2.14-548.g81bd20d6.el8.x86 4.1 MB/s | 6.1 MB 00:01 2024-02-20T12:45:56.468 INFO:teuthology.orchestra.run.smithi071.stdout:(2/23): ceph-selinux-16.2.14-548.g81bd20d6.el8. 309 kB/s | 26 kB 00:00 2024-02-20T12:45:56.794 INFO:teuthology.orchestra.run.smithi071.stdout:(3/23): libcephfs2-16.2.14-548.g81bd20d6.el8.x8 2.5 MB/s | 825 kB 00:00 2024-02-20T12:45:56.944 INFO:teuthology.orchestra.run.smithi071.stdout:(4/23): libradosstriper1-16.2.14-548.g81bd20d6. 3.2 MB/s | 494 kB 00:00 2024-02-20T12:45:57.732 INFO:teuthology.orchestra.run.smithi071.stdout:(5/23): librgw2-16.2.14-548.g81bd20d6.el8.x86_6 4.8 MB/s | 3.8 MB 00:00 2024-02-20T12:45:57.787 INFO:teuthology.orchestra.run.smithi071.stdout:(6/23): python3-ceph-argparse-16.2.14-548.g81bd 775 kB/s | 45 kB 00:00 2024-02-20T12:45:57.856 INFO:teuthology.orchestra.run.smithi071.stdout:(7/23): python3-ceph-common-16.2.14-548.g81bd20 1.4 MB/s | 99 kB 00:00 2024-02-20T12:45:58.012 INFO:teuthology.orchestra.run.smithi071.stdout:(8/23): ceph-radosgw-16.2.14-548.g81bd20d6.el8. 3.7 MB/s | 12 MB 00:03 2024-02-20T12:45:58.037 INFO:teuthology.orchestra.run.smithi071.stdout:(9/23): python3-cephfs-16.2.14-548.g81bd20d6.el 1.1 MB/s | 215 kB 00:00 2024-02-20T12:45:58.137 INFO:teuthology.orchestra.run.smithi071.stdout:(10/23): python3-rbd-16.2.14-548.g81bd20d6.el8. 3.6 MB/s | 369 kB 00:00 2024-02-20T12:45:58.163 INFO:teuthology.orchestra.run.smithi071.stdout:(11/23): python3-rados-16.2.14-548.g81bd20d6.el 2.5 MB/s | 389 kB 00:00 2024-02-20T12:45:58.196 INFO:teuthology.orchestra.run.smithi071.stdout:(12/23): python3-rgw-16.2.14-548.g81bd20d6.el8. 1.9 MB/s | 114 kB 00:00 2024-02-20T12:45:58.216 INFO:teuthology.orchestra.run.smithi064.stdout:Dependencies resolved. 2024-02-20T12:45:58.218 INFO:teuthology.orchestra.run.smithi064.stdout:================================================================================ 2024-02-20T12:45:58.218 INFO:teuthology.orchestra.run.smithi064.stdout: Package Arch Version Repository Size 2024-02-20T12:45:58.218 INFO:teuthology.orchestra.run.smithi064.stdout:================================================================================ 2024-02-20T12:45:58.219 INFO:teuthology.orchestra.run.smithi064.stdout:Installing: 2024-02-20T12:45:58.219 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-radosgw x86_64 2:16.2.14-548.g81bd20d6.el8 ceph 12 M 2024-02-20T12:45:58.219 INFO:teuthology.orchestra.run.smithi064.stdout:Upgrading: 2024-02-20T12:45:58.219 INFO:teuthology.orchestra.run.smithi064.stdout: librados2 x86_64 2:16.2.14-548.g81bd20d6.el8 ceph 3.7 M 2024-02-20T12:45:58.219 INFO:teuthology.orchestra.run.smithi064.stdout: librbd1 x86_64 2:16.2.14-548.g81bd20d6.el8 ceph 4.2 M 2024-02-20T12:45:58.219 INFO:teuthology.orchestra.run.smithi064.stdout:Installing dependencies: 2024-02-20T12:45:58.219 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-base x86_64 2:16.2.14-548.g81bd20d6.el8 ceph 6.1 M 2024-02-20T12:45:58.219 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-common x86_64 2:16.2.14-548.g81bd20d6.el8 ceph 24 M 2024-02-20T12:45:58.219 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-selinux x86_64 2:16.2.14-548.g81bd20d6.el8 ceph 26 k 2024-02-20T12:45:58.219 INFO:teuthology.orchestra.run.smithi064.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2024-02-20T12:45:58.219 INFO:teuthology.orchestra.run.smithi064.stdout: leveldb x86_64 1.22-1.el8 epel 181 k 2024-02-20T12:45:58.219 INFO:teuthology.orchestra.run.smithi064.stdout: libcephfs2 x86_64 2:16.2.14-548.g81bd20d6.el8 ceph 825 k 2024-02-20T12:45:58.219 INFO:teuthology.orchestra.run.smithi064.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2024-02-20T12:45:58.219 INFO:teuthology.orchestra.run.smithi064.stdout: librabbitmq x86_64 0.9.0-5.el8 baseos 47 k 2024-02-20T12:45:58.220 INFO:teuthology.orchestra.run.smithi064.stdout: libradosstriper1 x86_64 2:16.2.14-548.g81bd20d6.el8 ceph 494 k 2024-02-20T12:45:58.220 INFO:teuthology.orchestra.run.smithi064.stdout: librdkafka x86_64 1.6.1-1.el8 appstream 677 k 2024-02-20T12:45:58.220 INFO:teuthology.orchestra.run.smithi064.stdout: librgw2 x86_64 2:16.2.14-548.g81bd20d6.el8 ceph 3.8 M 2024-02-20T12:45:58.220 INFO:teuthology.orchestra.run.smithi064.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2024-02-20T12:45:58.220 INFO:teuthology.orchestra.run.smithi064.stdout: lttng-ust x86_64 2.8.1-11.el8 appstream 259 k 2024-02-20T12:45:58.220 INFO:teuthology.orchestra.run.smithi064.stdout: python3-ceph-argparse x86_64 2:16.2.14-548.g81bd20d6.el8 ceph 45 k 2024-02-20T12:45:58.220 INFO:teuthology.orchestra.run.smithi064.stdout: python3-ceph-common x86_64 2:16.2.14-548.g81bd20d6.el8 ceph 99 k 2024-02-20T12:45:58.220 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cephfs x86_64 2:16.2.14-548.g81bd20d6.el8 ceph 215 k 2024-02-20T12:45:58.220 INFO:teuthology.orchestra.run.smithi064.stdout: python3-prettytable noarch 0.7.2-14.el8 appstream 44 k 2024-02-20T12:45:58.221 INFO:teuthology.orchestra.run.smithi064.stdout: python3-rados x86_64 2:16.2.14-548.g81bd20d6.el8 ceph 389 k 2024-02-20T12:45:58.221 INFO:teuthology.orchestra.run.smithi064.stdout: python3-rbd x86_64 2:16.2.14-548.g81bd20d6.el8 ceph 369 k 2024-02-20T12:45:58.221 INFO:teuthology.orchestra.run.smithi064.stdout: python3-rgw x86_64 2:16.2.14-548.g81bd20d6.el8 ceph 114 k 2024-02-20T12:45:58.221 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-20T12:45:58.221 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction Summary 2024-02-20T12:45:58.221 INFO:teuthology.orchestra.run.smithi064.stdout:================================================================================ 2024-02-20T12:45:58.221 INFO:teuthology.orchestra.run.smithi064.stdout:Install 21 Packages 2024-02-20T12:45:58.221 INFO:teuthology.orchestra.run.smithi064.stdout:Upgrade 2 Packages 2024-02-20T12:45:58.221 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-20T12:45:58.222 INFO:teuthology.orchestra.run.smithi064.stdout:Total download size: 58 M 2024-02-20T12:45:58.222 INFO:teuthology.orchestra.run.smithi064.stdout:Downloading Packages: 2024-02-20T12:45:58.671 INFO:teuthology.orchestra.run.smithi071.stdout:(13/23): lttng-ust-2.8.1-11.el8.x86_64.rpm 546 kB/s | 259 kB 00:00 2024-02-20T12:45:58.705 INFO:teuthology.orchestra.run.smithi071.stdout:(14/23): librdkafka-1.6.1-1.el8.x86_64.rpm 1.2 MB/s | 677 kB 00:00 2024-02-20T12:45:58.802 INFO:teuthology.orchestra.run.smithi071.stdout:(15/23): python3-prettytable-0.7.2-14.el8.noarc 336 kB/s | 44 kB 00:00 2024-02-20T12:45:58.831 INFO:teuthology.orchestra.run.smithi071.stdout:(16/23): librabbitmq-0.9.0-5.el8.x86_64.rpm 375 kB/s | 47 kB 00:00 2024-02-20T12:45:58.998 INFO:teuthology.orchestra.run.smithi071.stdout:(17/23): leveldb-1.22-1.el8.x86_64.rpm 1.1 MB/s | 181 kB 00:00 2024-02-20T12:45:59.023 INFO:teuthology.orchestra.run.smithi071.stdout:(18/23): gperftools-libs-2.7-9.el8.x86_64.rpm 1.4 MB/s | 306 kB 00:00 2024-02-20T12:45:59.057 INFO:teuthology.orchestra.run.smithi071.stdout:(19/23): liboath-2.6.2-3.el8.x86_64.rpm 1.0 MB/s | 59 kB 00:00 2024-02-20T12:45:59.082 INFO:teuthology.orchestra.run.smithi071.stdout:(20/23): libunwind-1.3.1-3.el8.x86_64.rpm 1.3 MB/s | 75 kB 00:00 2024-02-20T12:45:59.383 INFO:teuthology.orchestra.run.smithi064.stdout:(1/23): ceph-base-16.2.14-548.g81bd20d6.el8.x86 8.1 MB/s | 6.1 MB 00:00 2024-02-20T12:45:59.441 INFO:teuthology.orchestra.run.smithi064.stdout:(2/23): ceph-selinux-16.2.14-548.g81bd20d6.el8. 441 kB/s | 26 kB 00:00 2024-02-20T12:45:59.567 INFO:teuthology.orchestra.run.smithi064.stdout:(3/23): libcephfs2-16.2.14-548.g81bd20d6.el8.x8 6.5 MB/s | 825 kB 00:00 2024-02-20T12:45:59.641 INFO:teuthology.orchestra.run.smithi064.stdout:(4/23): libradosstriper1-16.2.14-548.g81bd20d6. 6.4 MB/s | 494 kB 00:00 2024-02-20T12:46:00.157 INFO:teuthology.orchestra.run.smithi071.stdout:(21/23): librados2-16.2.14-548.g81bd20d6.el8.x8 3.4 MB/s | 3.7 MB 00:01 2024-02-20T12:46:00.492 INFO:teuthology.orchestra.run.smithi064.stdout:(5/23): librgw2-16.2.14-548.g81bd20d6.el8.x86_6 4.5 MB/s | 3.8 MB 00:00 2024-02-20T12:46:00.565 INFO:teuthology.orchestra.run.smithi071.stdout:(22/23): librbd1-16.2.14-548.g81bd20d6.el8.x86_ 2.8 MB/s | 4.2 MB 00:01 2024-02-20T12:46:00.794 INFO:teuthology.orchestra.run.smithi064.stdout:(6/23): python3-ceph-argparse-16.2.14-548.g81bd 150 kB/s | 45 kB 00:00 2024-02-20T12:46:00.960 INFO:teuthology.orchestra.run.smithi064.stdout:(7/23): ceph-radosgw-16.2.14-548.g81bd20d6.el8. 5.0 MB/s | 12 MB 00:02 2024-02-20T12:46:00.985 INFO:teuthology.orchestra.run.smithi064.stdout:(8/23): python3-ceph-common-16.2.14-548.g81bd20 516 kB/s | 99 kB 00:00 2024-02-20T12:46:01.058 INFO:teuthology.orchestra.run.smithi064.stdout:(9/23): python3-cephfs-16.2.14-548.g81bd20d6.el 2.2 MB/s | 215 kB 00:00 2024-02-20T12:46:01.094 INFO:teuthology.orchestra.run.smithi064.stdout:(10/23): python3-rados-16.2.14-548.g81bd20d6.el 3.5 MB/s | 389 kB 00:00 2024-02-20T12:46:01.136 INFO:teuthology.orchestra.run.smithi064.stdout:(11/23): python3-rbd-16.2.14-548.g81bd20d6.el8. 4.6 MB/s | 369 kB 00:00 2024-02-20T12:46:01.162 INFO:teuthology.orchestra.run.smithi064.stdout:(12/23): python3-rgw-16.2.14-548.g81bd20d6.el8. 1.7 MB/s | 114 kB 00:00 2024-02-20T12:46:01.442 INFO:teuthology.orchestra.run.smithi071.stdout:(23/23): ceph-common-16.2.14-548.g81bd20d6.el8. 3.7 MB/s | 24 MB 00:06 2024-02-20T12:46:01.445 INFO:teuthology.orchestra.run.smithi071.stdout:-------------------------------------------------------------------------------- 2024-02-20T12:46:01.445 INFO:teuthology.orchestra.run.smithi071.stdout:Total 8.6 MB/s | 58 MB 00:06 2024-02-20T12:46:01.486 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction check 2024-02-20T12:46:01.533 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction check succeeded. 2024-02-20T12:46:01.533 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction test 2024-02-20T12:46:01.637 INFO:teuthology.orchestra.run.smithi064.stdout:(13/23): ceph-common-16.2.14-548.g81bd20d6.el8. 8.1 MB/s | 24 MB 00:03 2024-02-20T12:46:01.837 INFO:teuthology.orchestra.run.smithi064.stdout:(14/23): lttng-ust-2.8.1-11.el8.x86_64.rpm 384 kB/s | 259 kB 00:00 2024-02-20T12:46:01.885 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction test succeeded. 2024-02-20T12:46:01.887 INFO:teuthology.orchestra.run.smithi064.stdout:(15/23): librdkafka-1.6.1-1.el8.x86_64.rpm 901 kB/s | 677 kB 00:00 2024-02-20T12:46:01.888 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction 2024-02-20T12:46:01.904 INFO:teuthology.orchestra.run.smithi064.stdout:(16/23): librabbitmq-0.9.0-5.el8.x86_64.rpm 704 kB/s | 47 kB 00:00 2024-02-20T12:46:01.946 INFO:teuthology.orchestra.run.smithi064.stdout:(17/23): python3-prettytable-0.7.2-14.el8.noarc 142 kB/s | 44 kB 00:00 2024-02-20T12:46:02.063 INFO:teuthology.orchestra.run.smithi064.stdout:(18/23): leveldb-1.22-1.el8.x86_64.rpm 1.1 MB/s | 181 kB 00:00 2024-02-20T12:46:02.080 INFO:teuthology.orchestra.run.smithi064.stdout:(19/23): liboath-2.6.2-3.el8.x86_64.rpm 442 kB/s | 59 kB 00:00 2024-02-20T12:46:02.106 INFO:teuthology.orchestra.run.smithi064.stdout:(20/23): gperftools-libs-2.7-9.el8.x86_64.rpm 1.4 MB/s | 306 kB 00:00 2024-02-20T12:46:02.139 INFO:teuthology.orchestra.run.smithi064.stdout:(21/23): libunwind-1.3.1-3.el8.x86_64.rpm 995 kB/s | 75 kB 00:00 2024-02-20T12:46:02.489 INFO:teuthology.orchestra.run.smithi064.stdout:(22/23): librados2-16.2.14-548.g81bd20d6.el8.x8 9.2 MB/s | 3.7 MB 00:00 2024-02-20T12:46:02.548 INFO:teuthology.orchestra.run.smithi064.stdout:(23/23): librbd1-16.2.14-548.g81bd20d6.el8.x86_ 9.5 MB/s | 4.2 MB 00:00 2024-02-20T12:46:02.553 INFO:teuthology.orchestra.run.smithi064.stdout:-------------------------------------------------------------------------------- 2024-02-20T12:46:02.553 INFO:teuthology.orchestra.run.smithi064.stdout:Total 13 MB/s | 58 MB 00:04 2024-02-20T12:46:02.599 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction check 2024-02-20T12:46:02.651 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction check succeeded. 2024-02-20T12:46:02.651 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction test 2024-02-20T12:46:02.930 INFO:teuthology.orchestra.run.smithi071.stdout: Preparing : 1/1 2024-02-20T12:46:03.032 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction test succeeded. 2024-02-20T12:46:03.035 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction 2024-02-20T12:46:03.139 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : liboath-2.6.2-3.el8.x86_64 1/25 2024-02-20T12:46:03.372 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 2/25 2024-02-20T12:46:03.752 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 2/25 2024-02-20T12:46:03.964 INFO:teuthology.orchestra.run.smithi071.stdout: Upgrading : librados2-2:16.2.14-548.g81bd20d6.el8.x86_64 3/25 2024-02-20T12:46:04.057 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: librados2-2:16.2.14-548.g81bd20d6.el8.x86_64 3/25 2024-02-20T12:46:04.131 INFO:teuthology.orchestra.run.smithi064.stdout: Preparing : 1/1 2024-02-20T12:46:04.345 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-rados-2:16.2.14-548.g81bd20d6.el8.x86_64 4/25 2024-02-20T12:46:04.383 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : liboath-2.6.2-3.el8.x86_64 1/25 2024-02-20T12:46:04.564 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : libcephfs2-2:16.2.14-548.g81bd20d6.el8.x86_64 5/25 2024-02-20T12:46:04.580 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 2/25 2024-02-20T12:46:04.974 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: libcephfs2-2:16.2.14-548.g81bd20d6.el8.x86_64 5/25 2024-02-20T12:46:04.991 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 2/25 2024-02-20T12:46:05.155 INFO:teuthology.orchestra.run.smithi071.stdout: Upgrading : librbd1-2:16.2.14-548.g81bd20d6.el8.x86_64 6/25 2024-02-20T12:46:05.205 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: librbd1-2:16.2.14-548.g81bd20d6.el8.x86_64 6/25 2024-02-20T12:46:05.222 INFO:teuthology.orchestra.run.smithi064.stdout: Upgrading : librados2-2:16.2.14-548.g81bd20d6.el8.x86_64 3/25 2024-02-20T12:46:05.325 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: librados2-2:16.2.14-548.g81bd20d6.el8.x86_64 3/25 2024-02-20T12:46:05.448 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : librabbitmq-0.9.0-5.el8.x86_64 7/25 2024-02-20T12:46:05.741 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-rados-2:16.2.14-548.g81bd20d6.el8.x86_64 4/25 2024-02-20T12:46:05.947 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : libcephfs2-2:16.2.14-548.g81bd20d6.el8.x86_64 5/25 2024-02-20T12:46:05.954 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : librdkafka-1.6.1-1.el8.x86_64 8/25 2024-02-20T12:46:06.138 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : librgw2-2:16.2.14-548.g81bd20d6.el8.x86_64 9/25 2024-02-20T12:46:06.236 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: librgw2-2:16.2.14-548.g81bd20d6.el8.x86_64 9/25 2024-02-20T12:46:06.419 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : leveldb-1.22-1.el8.x86_64 10/25 2024-02-20T12:46:06.439 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: libcephfs2-2:16.2.14-548.g81bd20d6.el8.x86_64 5/25 2024-02-20T12:46:06.619 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-ceph-argparse-2:16.2.14-548.g81bd20d6.el8. 11/25 2024-02-20T12:46:06.664 INFO:teuthology.orchestra.run.smithi064.stdout: Upgrading : librbd1-2:16.2.14-548.g81bd20d6.el8.x86_64 6/25 2024-02-20T12:46:06.754 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: librbd1-2:16.2.14-548.g81bd20d6.el8.x86_64 6/25 2024-02-20T12:46:06.791 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-cephfs-2:16.2.14-548.g81bd20d6.el8.x86_64 12/25 2024-02-20T12:46:06.994 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : librabbitmq-0.9.0-5.el8.x86_64 7/25 2024-02-20T12:46:07.031 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-rgw-2:16.2.14-548.g81bd20d6.el8.x86_64 13/25 2024-02-20T12:46:07.254 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-rbd-2:16.2.14-548.g81bd20d6.el8.x86_64 14/25 2024-02-20T12:46:07.438 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : libradosstriper1-2:16.2.14-548.g81bd20d6.el8.x86_6 15/25 2024-02-20T12:46:07.508 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: libradosstriper1-2:16.2.14-548.g81bd20d6.el8.x86_6 15/25 2024-02-20T12:46:07.640 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : librdkafka-1.6.1-1.el8.x86_64 8/25 2024-02-20T12:46:07.790 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 16/25 2024-02-20T12:46:07.830 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : librgw2-2:16.2.14-548.g81bd20d6.el8.x86_64 9/25 2024-02-20T12:46:07.894 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: librgw2-2:16.2.14-548.g81bd20d6.el8.x86_64 9/25 2024-02-20T12:46:07.955 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 17/25 2024-02-20T12:46:08.221 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : leveldb-1.22-1.el8.x86_64 10/25 2024-02-20T12:46:08.221 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 18/25 2024-02-20T12:46:08.295 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-ceph-argparse-2:16.2.14-548.g81bd20d6.el8. 11/25 2024-02-20T12:46:08.355 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-ceph-common-2:16.2.14-548.g81bd20d6.el8.x8 19/25 2024-02-20T12:46:08.534 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-cephfs-2:16.2.14-548.g81bd20d6.el8.x86_64 12/25 2024-02-20T12:46:08.768 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-rgw-2:16.2.14-548.g81bd20d6.el8.x86_64 13/25 2024-02-20T12:46:09.008 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-rbd-2:16.2.14-548.g81bd20d6.el8.x86_64 14/25 2024-02-20T12:46:09.171 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : libradosstriper1-2:16.2.14-548.g81bd20d6.el8.x86_6 15/25 2024-02-20T12:46:09.240 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: libradosstriper1-2:16.2.14-548.g81bd20d6.el8.x86_6 15/25 2024-02-20T12:46:09.532 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 16/25 2024-02-20T12:46:09.740 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 17/25 2024-02-20T12:46:09.965 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 18/25 2024-02-20T12:46:10.205 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-ceph-common-2:16.2.14-548.g81bd20d6.el8.x8 19/25 2024-02-20T12:46:10.867 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: ceph-common-2:16.2.14-548.g81bd20d6.el8.x86_64 20/25 2024-02-20T12:46:11.113 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-common-2:16.2.14-548.g81bd20d6.el8.x86_64 20/25 2024-02-20T12:46:11.891 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: ceph-common-2:16.2.14-548.g81bd20d6.el8.x86_64 20/25 2024-02-20T12:46:12.226 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-base-2:16.2.14-548.g81bd20d6.el8.x86_64 21/25 2024-02-20T12:46:12.327 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: ceph-base-2:16.2.14-548.g81bd20d6.el8.x86_64 21/25 2024-02-20T12:46:12.487 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-selinux-2:16.2.14-548.g81bd20d6.el8.x86_64 22/25 2024-02-20T12:46:12.896 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-common-2:16.2.14-548.g81bd20d6.el8.x86_64 20/25 2024-02-20T12:46:13.197 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-common-2:16.2.14-548.g81bd20d6.el8.x86_64 20/25 2024-02-20T12:46:13.958 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-common-2:16.2.14-548.g81bd20d6.el8.x86_64 20/25 2024-02-20T12:46:14.296 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-base-2:16.2.14-548.g81bd20d6.el8.x86_64 21/25 2024-02-20T12:46:14.390 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-base-2:16.2.14-548.g81bd20d6.el8.x86_64 21/25 2024-02-20T12:46:14.562 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-selinux-2:16.2.14-548.g81bd20d6.el8.x86_64 22/25 2024-02-20T12:46:31.703 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: ceph-selinux-2:16.2.14-548.g81bd20d6.el8.x86_64 22/25 2024-02-20T12:46:31.704 INFO:teuthology.orchestra.run.smithi071.stdout:skipping the directory /sys 2024-02-20T12:46:31.704 INFO:teuthology.orchestra.run.smithi071.stdout:skipping the directory /proc 2024-02-20T12:46:31.704 INFO:teuthology.orchestra.run.smithi071.stdout:skipping the directory /dev 2024-02-20T12:46:31.704 INFO:teuthology.orchestra.run.smithi071.stdout:skipping the directory /run 2024-02-20T12:46:31.704 INFO:teuthology.orchestra.run.smithi071.stdout:skipping the directory /mnt 2024-02-20T12:46:31.704 INFO:teuthology.orchestra.run.smithi071.stdout:skipping the directory /var/tmp 2024-02-20T12:46:31.704 INFO:teuthology.orchestra.run.smithi071.stdout:skipping the directory /home 2024-02-20T12:46:31.704 INFO:teuthology.orchestra.run.smithi071.stdout:skipping the directory /tmp 2024-02-20T12:46:31.704 INFO:teuthology.orchestra.run.smithi071.stdout:skipping the directory /dev 2024-02-20T12:46:31.704 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-20T12:46:32.863 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-radosgw-2:16.2.14-548.g81bd20d6.el8.x86_64 23/25 2024-02-20T12:46:32.950 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: ceph-radosgw-2:16.2.14-548.g81bd20d6.el8.x86_64 23/25 2024-02-20T12:46:32.950 INFO:teuthology.orchestra.run.smithi071.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 24/25 2024-02-20T12:46:33.193 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 24/25 2024-02-20T12:46:33.193 INFO:teuthology.orchestra.run.smithi071.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 25/25 2024-02-20T12:46:33.893 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 25/25 2024-02-20T12:46:33.894 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-base-2:16.2.14-548.g81bd20d6.el8.x86_64 1/25 2024-02-20T12:46:33.894 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-common-2:16.2.14-548.g81bd20d6.el8.x86_64 2/25 2024-02-20T12:46:33.894 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-radosgw-2:16.2.14-548.g81bd20d6.el8.x86_64 3/25 2024-02-20T12:46:33.894 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-selinux-2:16.2.14-548.g81bd20d6.el8.x86_64 4/25 2024-02-20T12:46:33.894 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : libcephfs2-2:16.2.14-548.g81bd20d6.el8.x86_64 5/25 2024-02-20T12:46:33.894 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : libradosstriper1-2:16.2.14-548.g81bd20d6.el8.x86_6 6/25 2024-02-20T12:46:33.894 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : librgw2-2:16.2.14-548.g81bd20d6.el8.x86_64 7/25 2024-02-20T12:46:33.894 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-ceph-argparse-2:16.2.14-548.g81bd20d6.el8. 8/25 2024-02-20T12:46:33.894 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-ceph-common-2:16.2.14-548.g81bd20d6.el8.x8 9/25 2024-02-20T12:46:33.894 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-cephfs-2:16.2.14-548.g81bd20d6.el8.x86_64 10/25 2024-02-20T12:46:33.894 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-rados-2:16.2.14-548.g81bd20d6.el8.x86_64 11/25 2024-02-20T12:46:33.894 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-rbd-2:16.2.14-548.g81bd20d6.el8.x86_64 12/25 2024-02-20T12:46:33.894 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-rgw-2:16.2.14-548.g81bd20d6.el8.x86_64 13/25 2024-02-20T12:46:33.895 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : librdkafka-1.6.1-1.el8.x86_64 14/25 2024-02-20T12:46:33.895 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 15/25 2024-02-20T12:46:33.895 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 16/25 2024-02-20T12:46:33.895 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : librabbitmq-0.9.0-5.el8.x86_64 17/25 2024-02-20T12:46:33.895 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 18/25 2024-02-20T12:46:33.895 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : leveldb-1.22-1.el8.x86_64 19/25 2024-02-20T12:46:33.895 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 20/25 2024-02-20T12:46:33.895 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 21/25 2024-02-20T12:46:33.895 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : librados2-2:16.2.14-548.g81bd20d6.el8.x86_64 22/25 2024-02-20T12:46:33.895 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 23/25 2024-02-20T12:46:33.895 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : librbd1-2:16.2.14-548.g81bd20d6.el8.x86_64 24/25 2024-02-20T12:46:34.505 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-selinux-2:16.2.14-548.g81bd20d6.el8.x86_64 22/25 2024-02-20T12:46:34.505 INFO:teuthology.orchestra.run.smithi064.stdout:skipping the directory /sys 2024-02-20T12:46:34.505 INFO:teuthology.orchestra.run.smithi064.stdout:skipping the directory /proc 2024-02-20T12:46:34.505 INFO:teuthology.orchestra.run.smithi064.stdout:skipping the directory /dev 2024-02-20T12:46:34.505 INFO:teuthology.orchestra.run.smithi064.stdout:skipping the directory /run 2024-02-20T12:46:34.505 INFO:teuthology.orchestra.run.smithi064.stdout:skipping the directory /mnt 2024-02-20T12:46:34.505 INFO:teuthology.orchestra.run.smithi064.stdout:skipping the directory /var/tmp 2024-02-20T12:46:34.506 INFO:teuthology.orchestra.run.smithi064.stdout:skipping the directory /home 2024-02-20T12:46:34.506 INFO:teuthology.orchestra.run.smithi064.stdout:skipping the directory /tmp 2024-02-20T12:46:34.506 INFO:teuthology.orchestra.run.smithi064.stdout:skipping the directory /dev 2024-02-20T12:46:34.506 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-20T12:46:34.752 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 25/25 2024-02-20T12:46:34.753 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-20T12:46:34.753 INFO:teuthology.orchestra.run.smithi071.stdout:Upgraded: 2024-02-20T12:46:34.753 INFO:teuthology.orchestra.run.smithi071.stdout: librados2-2:16.2.14-548.g81bd20d6.el8.x86_64 2024-02-20T12:46:34.753 INFO:teuthology.orchestra.run.smithi071.stdout: librbd1-2:16.2.14-548.g81bd20d6.el8.x86_64 2024-02-20T12:46:34.753 INFO:teuthology.orchestra.run.smithi071.stdout:Installed: 2024-02-20T12:46:34.753 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-base-2:16.2.14-548.g81bd20d6.el8.x86_64 2024-02-20T12:46:34.753 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-common-2:16.2.14-548.g81bd20d6.el8.x86_64 2024-02-20T12:46:34.753 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-radosgw-2:16.2.14-548.g81bd20d6.el8.x86_64 2024-02-20T12:46:34.753 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-selinux-2:16.2.14-548.g81bd20d6.el8.x86_64 2024-02-20T12:46:34.753 INFO:teuthology.orchestra.run.smithi071.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2024-02-20T12:46:34.753 INFO:teuthology.orchestra.run.smithi071.stdout: leveldb-1.22-1.el8.x86_64 2024-02-20T12:46:34.753 INFO:teuthology.orchestra.run.smithi071.stdout: libcephfs2-2:16.2.14-548.g81bd20d6.el8.x86_64 2024-02-20T12:46:34.753 INFO:teuthology.orchestra.run.smithi071.stdout: liboath-2.6.2-3.el8.x86_64 2024-02-20T12:46:34.754 INFO:teuthology.orchestra.run.smithi071.stdout: librabbitmq-0.9.0-5.el8.x86_64 2024-02-20T12:46:34.754 INFO:teuthology.orchestra.run.smithi071.stdout: libradosstriper1-2:16.2.14-548.g81bd20d6.el8.x86_64 2024-02-20T12:46:34.754 INFO:teuthology.orchestra.run.smithi071.stdout: librdkafka-1.6.1-1.el8.x86_64 2024-02-20T12:46:34.754 INFO:teuthology.orchestra.run.smithi071.stdout: librgw2-2:16.2.14-548.g81bd20d6.el8.x86_64 2024-02-20T12:46:34.754 INFO:teuthology.orchestra.run.smithi071.stdout: libunwind-1.3.1-3.el8.x86_64 2024-02-20T12:46:34.754 INFO:teuthology.orchestra.run.smithi071.stdout: lttng-ust-2.8.1-11.el8.x86_64 2024-02-20T12:46:34.754 INFO:teuthology.orchestra.run.smithi071.stdout: python3-ceph-argparse-2:16.2.14-548.g81bd20d6.el8.x86_64 2024-02-20T12:46:34.754 INFO:teuthology.orchestra.run.smithi071.stdout: python3-ceph-common-2:16.2.14-548.g81bd20d6.el8.x86_64 2024-02-20T12:46:34.754 INFO:teuthology.orchestra.run.smithi071.stdout: python3-cephfs-2:16.2.14-548.g81bd20d6.el8.x86_64 2024-02-20T12:46:34.754 INFO:teuthology.orchestra.run.smithi071.stdout: python3-prettytable-0.7.2-14.el8.noarch 2024-02-20T12:46:34.754 INFO:teuthology.orchestra.run.smithi071.stdout: python3-rados-2:16.2.14-548.g81bd20d6.el8.x86_64 2024-02-20T12:46:34.754 INFO:teuthology.orchestra.run.smithi071.stdout: python3-rbd-2:16.2.14-548.g81bd20d6.el8.x86_64 2024-02-20T12:46:34.754 INFO:teuthology.orchestra.run.smithi071.stdout: python3-rgw-2:16.2.14-548.g81bd20d6.el8.x86_64 2024-02-20T12:46:34.755 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-20T12:46:34.755 INFO:teuthology.orchestra.run.smithi071.stdout:Complete! 2024-02-20T12:46:35.020 DEBUG:teuthology.orchestra.run.smithi071:> sudo yum -y install ceph-test 2024-02-20T12:46:35.776 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-radosgw-2:16.2.14-548.g81bd20d6.el8.x86_64 23/25 2024-02-20T12:46:35.777 INFO:teuthology.orchestra.run.smithi071.stdout:Last metadata expiration check: 0:00:44 ago on Tue 20 Feb 2024 12:45:51 PM UTC. 2024-02-20T12:46:35.880 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-radosgw-2:16.2.14-548.g81bd20d6.el8.x86_64 23/25 2024-02-20T12:46:35.880 INFO:teuthology.orchestra.run.smithi064.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 24/25 2024-02-20T12:46:36.245 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 24/25 2024-02-20T12:46:36.246 INFO:teuthology.orchestra.run.smithi064.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 25/25 2024-02-20T12:46:37.014 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 25/25 2024-02-20T12:46:37.015 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-base-2:16.2.14-548.g81bd20d6.el8.x86_64 1/25 2024-02-20T12:46:37.015 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-common-2:16.2.14-548.g81bd20d6.el8.x86_64 2/25 2024-02-20T12:46:37.015 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-radosgw-2:16.2.14-548.g81bd20d6.el8.x86_64 3/25 2024-02-20T12:46:37.015 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-selinux-2:16.2.14-548.g81bd20d6.el8.x86_64 4/25 2024-02-20T12:46:37.015 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libcephfs2-2:16.2.14-548.g81bd20d6.el8.x86_64 5/25 2024-02-20T12:46:37.015 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libradosstriper1-2:16.2.14-548.g81bd20d6.el8.x86_6 6/25 2024-02-20T12:46:37.015 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : librgw2-2:16.2.14-548.g81bd20d6.el8.x86_64 7/25 2024-02-20T12:46:37.015 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-ceph-argparse-2:16.2.14-548.g81bd20d6.el8. 8/25 2024-02-20T12:46:37.015 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-ceph-common-2:16.2.14-548.g81bd20d6.el8.x8 9/25 2024-02-20T12:46:37.016 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-cephfs-2:16.2.14-548.g81bd20d6.el8.x86_64 10/25 2024-02-20T12:46:37.016 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-rados-2:16.2.14-548.g81bd20d6.el8.x86_64 11/25 2024-02-20T12:46:37.016 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-rbd-2:16.2.14-548.g81bd20d6.el8.x86_64 12/25 2024-02-20T12:46:37.016 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-rgw-2:16.2.14-548.g81bd20d6.el8.x86_64 13/25 2024-02-20T12:46:37.016 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : librdkafka-1.6.1-1.el8.x86_64 14/25 2024-02-20T12:46:37.016 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 15/25 2024-02-20T12:46:37.016 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 16/25 2024-02-20T12:46:37.016 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : librabbitmq-0.9.0-5.el8.x86_64 17/25 2024-02-20T12:46:37.016 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 18/25 2024-02-20T12:46:37.016 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : leveldb-1.22-1.el8.x86_64 19/25 2024-02-20T12:46:37.017 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 20/25 2024-02-20T12:46:37.017 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 21/25 2024-02-20T12:46:37.017 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : librados2-2:16.2.14-548.g81bd20d6.el8.x86_64 22/25 2024-02-20T12:46:37.017 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 23/25 2024-02-20T12:46:37.017 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : librbd1-2:16.2.14-548.g81bd20d6.el8.x86_64 24/25 2024-02-20T12:46:37.300 INFO:teuthology.orchestra.run.smithi071.stdout:Dependencies resolved. 2024-02-20T12:46:37.301 INFO:teuthology.orchestra.run.smithi071.stdout:================================================================================ 2024-02-20T12:46:37.301 INFO:teuthology.orchestra.run.smithi071.stdout: Package Arch Version Repository Size 2024-02-20T12:46:37.301 INFO:teuthology.orchestra.run.smithi071.stdout:================================================================================ 2024-02-20T12:46:37.301 INFO:teuthology.orchestra.run.smithi071.stdout:Installing: 2024-02-20T12:46:37.302 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-test x86_64 2:16.2.14-548.g81bd20d6.el8 ceph 52 M 2024-02-20T12:46:37.302 INFO:teuthology.orchestra.run.smithi071.stdout:Installing dependencies: 2024-02-20T12:46:37.302 INFO:teuthology.orchestra.run.smithi071.stdout: jq x86_64 1.6-8.el8 appstream 203 k 2024-02-20T12:46:37.302 INFO:teuthology.orchestra.run.smithi071.stdout: libcephsqlite x86_64 2:16.2.14-548.g81bd20d6.el8 ceph 174 k 2024-02-20T12:46:37.302 INFO:teuthology.orchestra.run.smithi071.stdout: oniguruma x86_64 6.8.2-3.el8 appstream 188 k 2024-02-20T12:46:37.302 INFO:teuthology.orchestra.run.smithi071.stdout: socat x86_64 1.7.4.1-1.el8 appstream 323 k 2024-02-20T12:46:37.302 INFO:teuthology.orchestra.run.smithi071.stdout: xmlstarlet x86_64 1.6.1-20.el8 appstream 71 k 2024-02-20T12:46:37.302 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-20T12:46:37.302 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction Summary 2024-02-20T12:46:37.302 INFO:teuthology.orchestra.run.smithi071.stdout:================================================================================ 2024-02-20T12:46:37.302 INFO:teuthology.orchestra.run.smithi071.stdout:Install 6 Packages 2024-02-20T12:46:37.302 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-20T12:46:37.303 INFO:teuthology.orchestra.run.smithi071.stdout:Total download size: 53 M 2024-02-20T12:46:37.303 INFO:teuthology.orchestra.run.smithi071.stdout:Installed size: 197 M 2024-02-20T12:46:37.303 INFO:teuthology.orchestra.run.smithi071.stdout:Downloading Packages: 2024-02-20T12:46:37.824 INFO:teuthology.orchestra.run.smithi071.stdout:(1/6): libcephsqlite-16.2.14-548.g81bd20d6.el8. 532 kB/s | 174 kB 00:00 2024-02-20T12:46:37.930 INFO:teuthology.orchestra.run.smithi071.stdout:(2/6): jq-1.6-8.el8.x86_64.rpm 469 kB/s | 203 kB 00:00 2024-02-20T12:46:37.995 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 25/25 2024-02-20T12:46:37.995 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-20T12:46:37.995 INFO:teuthology.orchestra.run.smithi064.stdout:Upgraded: 2024-02-20T12:46:37.996 INFO:teuthology.orchestra.run.smithi064.stdout: librados2-2:16.2.14-548.g81bd20d6.el8.x86_64 2024-02-20T12:46:37.996 INFO:teuthology.orchestra.run.smithi064.stdout: librbd1-2:16.2.14-548.g81bd20d6.el8.x86_64 2024-02-20T12:46:37.996 INFO:teuthology.orchestra.run.smithi064.stdout:Installed: 2024-02-20T12:46:37.996 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-base-2:16.2.14-548.g81bd20d6.el8.x86_64 2024-02-20T12:46:37.996 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-common-2:16.2.14-548.g81bd20d6.el8.x86_64 2024-02-20T12:46:37.996 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-radosgw-2:16.2.14-548.g81bd20d6.el8.x86_64 2024-02-20T12:46:37.996 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-selinux-2:16.2.14-548.g81bd20d6.el8.x86_64 2024-02-20T12:46:37.996 INFO:teuthology.orchestra.run.smithi064.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2024-02-20T12:46:37.996 INFO:teuthology.orchestra.run.smithi064.stdout: leveldb-1.22-1.el8.x86_64 2024-02-20T12:46:37.996 INFO:teuthology.orchestra.run.smithi064.stdout: libcephfs2-2:16.2.14-548.g81bd20d6.el8.x86_64 2024-02-20T12:46:37.996 INFO:teuthology.orchestra.run.smithi064.stdout: liboath-2.6.2-3.el8.x86_64 2024-02-20T12:46:37.996 INFO:teuthology.orchestra.run.smithi064.stdout: librabbitmq-0.9.0-5.el8.x86_64 2024-02-20T12:46:37.996 INFO:teuthology.orchestra.run.smithi064.stdout: libradosstriper1-2:16.2.14-548.g81bd20d6.el8.x86_64 2024-02-20T12:46:37.996 INFO:teuthology.orchestra.run.smithi064.stdout: librdkafka-1.6.1-1.el8.x86_64 2024-02-20T12:46:37.997 INFO:teuthology.orchestra.run.smithi064.stdout: librgw2-2:16.2.14-548.g81bd20d6.el8.x86_64 2024-02-20T12:46:37.997 INFO:teuthology.orchestra.run.smithi064.stdout: libunwind-1.3.1-3.el8.x86_64 2024-02-20T12:46:37.997 INFO:teuthology.orchestra.run.smithi064.stdout: lttng-ust-2.8.1-11.el8.x86_64 2024-02-20T12:46:37.997 INFO:teuthology.orchestra.run.smithi064.stdout: python3-ceph-argparse-2:16.2.14-548.g81bd20d6.el8.x86_64 2024-02-20T12:46:37.997 INFO:teuthology.orchestra.run.smithi064.stdout: python3-ceph-common-2:16.2.14-548.g81bd20d6.el8.x86_64 2024-02-20T12:46:37.997 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cephfs-2:16.2.14-548.g81bd20d6.el8.x86_64 2024-02-20T12:46:37.997 INFO:teuthology.orchestra.run.smithi064.stdout: python3-prettytable-0.7.2-14.el8.noarch 2024-02-20T12:46:37.997 INFO:teuthology.orchestra.run.smithi064.stdout: python3-rados-2:16.2.14-548.g81bd20d6.el8.x86_64 2024-02-20T12:46:37.997 INFO:teuthology.orchestra.run.smithi064.stdout: python3-rbd-2:16.2.14-548.g81bd20d6.el8.x86_64 2024-02-20T12:46:37.997 INFO:teuthology.orchestra.run.smithi064.stdout: python3-rgw-2:16.2.14-548.g81bd20d6.el8.x86_64 2024-02-20T12:46:37.997 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-20T12:46:37.997 INFO:teuthology.orchestra.run.smithi064.stdout:Complete! 2024-02-20T12:46:38.181 INFO:teuthology.orchestra.run.smithi071.stdout:(3/6): socat-1.7.4.1-1.el8.x86_64.rpm 1.3 MB/s | 323 kB 00:00 2024-02-20T12:46:38.256 INFO:teuthology.orchestra.run.smithi071.stdout:(4/6): oniguruma-6.8.2-3.el8.x86_64.rpm 436 kB/s | 188 kB 00:00 2024-02-20T12:46:38.281 INFO:teuthology.orchestra.run.smithi071.stdout:(5/6): xmlstarlet-1.6.1-20.el8.x86_64.rpm 705 kB/s | 71 kB 00:00 2024-02-20T12:46:38.355 DEBUG:teuthology.orchestra.run.smithi064:> sudo yum -y install ceph-test 2024-02-20T12:46:39.147 INFO:teuthology.orchestra.run.smithi064.stdout:Last metadata expiration check: 0:00:45 ago on Tue 20 Feb 2024 12:45:54 PM UTC. 2024-02-20T12:46:39.981 INFO:teuthology.orchestra.run.smithi071.stdout:(6/6): ceph-test-16.2.14-548.g81bd20d6.el8.x86_ 21 MB/s | 52 MB 00:02 2024-02-20T12:46:39.983 INFO:teuthology.orchestra.run.smithi071.stdout:-------------------------------------------------------------------------------- 2024-02-20T12:46:39.983 INFO:teuthology.orchestra.run.smithi071.stdout:Total 20 MB/s | 53 MB 00:02 2024-02-20T12:46:40.022 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction check 2024-02-20T12:46:40.042 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction check succeeded. 2024-02-20T12:46:40.042 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction test 2024-02-20T12:46:40.329 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction test succeeded. 2024-02-20T12:46:40.332 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction 2024-02-20T12:46:40.802 INFO:teuthology.orchestra.run.smithi071.stdout: Preparing : 1/1 2024-02-20T12:46:40.813 INFO:teuthology.orchestra.run.smithi064.stdout:Dependencies resolved. 2024-02-20T12:46:40.815 INFO:teuthology.orchestra.run.smithi064.stdout:================================================================================ 2024-02-20T12:46:40.815 INFO:teuthology.orchestra.run.smithi064.stdout: Package Arch Version Repository Size 2024-02-20T12:46:40.815 INFO:teuthology.orchestra.run.smithi064.stdout:================================================================================ 2024-02-20T12:46:40.815 INFO:teuthology.orchestra.run.smithi064.stdout:Installing: 2024-02-20T12:46:40.815 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-test x86_64 2:16.2.14-548.g81bd20d6.el8 ceph 52 M 2024-02-20T12:46:40.815 INFO:teuthology.orchestra.run.smithi064.stdout:Installing dependencies: 2024-02-20T12:46:40.815 INFO:teuthology.orchestra.run.smithi064.stdout: jq x86_64 1.6-8.el8 appstream 203 k 2024-02-20T12:46:40.815 INFO:teuthology.orchestra.run.smithi064.stdout: libcephsqlite x86_64 2:16.2.14-548.g81bd20d6.el8 ceph 174 k 2024-02-20T12:46:40.815 INFO:teuthology.orchestra.run.smithi064.stdout: oniguruma x86_64 6.8.2-3.el8 appstream 188 k 2024-02-20T12:46:40.815 INFO:teuthology.orchestra.run.smithi064.stdout: socat x86_64 1.7.4.1-1.el8 appstream 323 k 2024-02-20T12:46:40.816 INFO:teuthology.orchestra.run.smithi064.stdout: xmlstarlet x86_64 1.6.1-20.el8 appstream 71 k 2024-02-20T12:46:40.816 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-20T12:46:40.816 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction Summary 2024-02-20T12:46:40.816 INFO:teuthology.orchestra.run.smithi064.stdout:================================================================================ 2024-02-20T12:46:40.816 INFO:teuthology.orchestra.run.smithi064.stdout:Install 6 Packages 2024-02-20T12:46:40.816 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-20T12:46:40.816 INFO:teuthology.orchestra.run.smithi064.stdout:Total download size: 53 M 2024-02-20T12:46:40.817 INFO:teuthology.orchestra.run.smithi064.stdout:Installed size: 197 M 2024-02-20T12:46:40.817 INFO:teuthology.orchestra.run.smithi064.stdout:Downloading Packages: 2024-02-20T12:46:41.044 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 1/6 2024-02-20T12:46:41.053 INFO:teuthology.orchestra.run.smithi064.stdout:(1/6): libcephsqlite-16.2.14-548.g81bd20d6.el8. 899 kB/s | 174 kB 00:00 2024-02-20T12:46:41.270 INFO:teuthology.orchestra.run.smithi064.stdout:(2/6): jq-1.6-8.el8.x86_64.rpm 495 kB/s | 203 kB 00:00 2024-02-20T12:46:41.292 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 2/6 2024-02-20T12:46:41.456 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : oniguruma-6.8.2-3.el8.x86_64 3/6 2024-02-20T12:46:41.518 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: oniguruma-6.8.2-3.el8.x86_64 3/6 2024-02-20T12:46:41.589 INFO:teuthology.orchestra.run.smithi064.stdout:(3/6): socat-1.7.4.1-1.el8.x86_64.rpm 1.0 MB/s | 323 kB 00:00 2024-02-20T12:46:41.656 INFO:teuthology.orchestra.run.smithi064.stdout:(4/6): oniguruma-6.8.2-3.el8.x86_64.rpm 312 kB/s | 188 kB 00:00 2024-02-20T12:46:41.711 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : jq-1.6-8.el8.x86_64 4/6 2024-02-20T12:46:41.714 INFO:teuthology.orchestra.run.smithi064.stdout:(5/6): xmlstarlet-1.6.1-20.el8.x86_64.rpm 564 kB/s | 71 kB 00:00 2024-02-20T12:46:41.872 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : libcephsqlite-2:16.2.14-548.g81bd20d6.el8.x86_64 5/6 2024-02-20T12:46:43.388 INFO:teuthology.orchestra.run.smithi064.stdout:(6/6): ceph-test-16.2.14-548.g81bd20d6.el8.x86_ 21 MB/s | 52 MB 00:02 2024-02-20T12:46:43.390 INFO:teuthology.orchestra.run.smithi064.stdout:-------------------------------------------------------------------------------- 2024-02-20T12:46:43.391 INFO:teuthology.orchestra.run.smithi064.stdout:Total 21 MB/s | 53 MB 00:02 2024-02-20T12:46:43.435 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction check 2024-02-20T12:46:43.457 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction check succeeded. 2024-02-20T12:46:43.457 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction test 2024-02-20T12:46:43.761 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction test succeeded. 2024-02-20T12:46:43.765 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction 2024-02-20T12:46:44.290 INFO:teuthology.orchestra.run.smithi064.stdout: Preparing : 1/1 2024-02-20T12:46:44.574 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 1/6 2024-02-20T12:46:44.921 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 2/6 2024-02-20T12:46:45.083 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : oniguruma-6.8.2-3.el8.x86_64 3/6 2024-02-20T12:46:45.162 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: oniguruma-6.8.2-3.el8.x86_64 3/6 2024-02-20T12:46:45.408 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : jq-1.6-8.el8.x86_64 4/6 2024-02-20T12:46:45.608 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : libcephsqlite-2:16.2.14-548.g81bd20d6.el8.x86_64 5/6 2024-02-20T12:46:46.478 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: libcephsqlite-2:16.2.14-548.g81bd20d6.el8.x86_64 5/6 2024-02-20T12:46:46.711 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-test-2:16.2.14-548.g81bd20d6.el8.x86_64 6/6 2024-02-20T12:46:47.038 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: ceph-test-2:16.2.14-548.g81bd20d6.el8.x86_64 6/6 2024-02-20T12:46:47.038 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-test-2:16.2.14-548.g81bd20d6.el8.x86_64 1/6 2024-02-20T12:46:47.038 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : libcephsqlite-2:16.2.14-548.g81bd20d6.el8.x86_64 2/6 2024-02-20T12:46:47.038 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : jq-1.6-8.el8.x86_64 3/6 2024-02-20T12:46:47.038 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : oniguruma-6.8.2-3.el8.x86_64 4/6 2024-02-20T12:46:47.039 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 5/6 2024-02-20T12:46:47.474 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 6/6 2024-02-20T12:46:47.475 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-20T12:46:47.475 INFO:teuthology.orchestra.run.smithi071.stdout:Installed: 2024-02-20T12:46:47.475 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-test-2:16.2.14-548.g81bd20d6.el8.x86_64 2024-02-20T12:46:47.475 INFO:teuthology.orchestra.run.smithi071.stdout: jq-1.6-8.el8.x86_64 2024-02-20T12:46:47.475 INFO:teuthology.orchestra.run.smithi071.stdout: libcephsqlite-2:16.2.14-548.g81bd20d6.el8.x86_64 2024-02-20T12:46:47.475 INFO:teuthology.orchestra.run.smithi071.stdout: oniguruma-6.8.2-3.el8.x86_64 2024-02-20T12:46:47.475 INFO:teuthology.orchestra.run.smithi071.stdout: socat-1.7.4.1-1.el8.x86_64 2024-02-20T12:46:47.475 INFO:teuthology.orchestra.run.smithi071.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2024-02-20T12:46:47.475 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-20T12:46:47.475 INFO:teuthology.orchestra.run.smithi071.stdout:Complete! 2024-02-20T12:46:47.719 DEBUG:teuthology.orchestra.run.smithi071:> sudo yum -y install ceph 2024-02-20T12:46:48.472 INFO:teuthology.orchestra.run.smithi071.stdout:Last metadata expiration check: 0:00:57 ago on Tue 20 Feb 2024 12:45:51 PM UTC. 2024-02-20T12:46:50.023 INFO:teuthology.orchestra.run.smithi071.stdout:Dependencies resolved. 2024-02-20T12:46:50.027 INFO:teuthology.orchestra.run.smithi071.stdout:================================================================================ 2024-02-20T12:46:50.027 INFO:teuthology.orchestra.run.smithi071.stdout: Package Arch Version Repository Size 2024-02-20T12:46:50.027 INFO:teuthology.orchestra.run.smithi071.stdout:================================================================================ 2024-02-20T12:46:50.027 INFO:teuthology.orchestra.run.smithi071.stdout:Installing: 2024-02-20T12:46:50.027 INFO:teuthology.orchestra.run.smithi071.stdout: ceph x86_64 2:16.2.14-548.g81bd20d6.el8 ceph 6.4 k 2024-02-20T12:46:50.027 INFO:teuthology.orchestra.run.smithi071.stdout:Installing dependencies: 2024-02-20T12:46:50.027 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-mds x86_64 2:16.2.14-548.g81bd20d6.el8 ceph 2.4 M 2024-02-20T12:46:50.027 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-mgr x86_64 2:16.2.14-548.g81bd20d6.el8 ceph 1.6 M 2024-02-20T12:46:50.027 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-mgr-modules-core noarch 2:16.2.14-548.g81bd20d6.el8 ceph-noarch 245 k 2024-02-20T12:46:50.027 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-mon x86_64 2:16.2.14-548.g81bd20d6.el8 ceph 5.2 M 2024-02-20T12:46:50.027 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-osd x86_64 2:16.2.14-548.g81bd20d6.el8 ceph 19 M 2024-02-20T12:46:50.028 INFO:teuthology.orchestra.run.smithi071.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2024-02-20T12:46:50.028 INFO:teuthology.orchestra.run.smithi071.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2024-02-20T12:46:50.028 INFO:teuthology.orchestra.run.smithi071.stdout: python3-cffi x86_64 1.11.5-6.el8 baseos 249 k 2024-02-20T12:46:50.028 INFO:teuthology.orchestra.run.smithi071.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2024-02-20T12:46:50.028 INFO:teuthology.orchestra.run.smithi071.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2024-02-20T12:46:50.028 INFO:teuthology.orchestra.run.smithi071.stdout: python3-cryptography x86_64 3.2.1-7.el8 baseos 641 k 2024-02-20T12:46:50.028 INFO:teuthology.orchestra.run.smithi071.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2024-02-20T12:46:50.028 INFO:teuthology.orchestra.run.smithi071.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2024-02-20T12:46:50.028 INFO:teuthology.orchestra.run.smithi071.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2024-02-20T12:46:50.028 INFO:teuthology.orchestra.run.smithi071.stdout: python3-mako noarch 1.0.6-14.el8 appstream 157 k 2024-02-20T12:46:50.028 INFO:teuthology.orchestra.run.smithi071.stdout: python3-markupsafe x86_64 0.23-19.el8 appstream 39 k 2024-02-20T12:46:50.028 INFO:teuthology.orchestra.run.smithi071.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2024-02-20T12:46:50.028 INFO:teuthology.orchestra.run.smithi071.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2024-02-20T12:46:50.029 INFO:teuthology.orchestra.run.smithi071.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2024-02-20T12:46:50.029 INFO:teuthology.orchestra.run.smithi071.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 appstream 103 k 2024-02-20T12:46:50.029 INFO:teuthology.orchestra.run.smithi071.stdout: python3-pycparser noarch 2.14-14.el8 baseos 109 k 2024-02-20T12:46:50.029 INFO:teuthology.orchestra.run.smithi071.stdout: python3-pytz noarch 2017.2-11.el8 appstream 57 k 2024-02-20T12:46:50.029 INFO:teuthology.orchestra.run.smithi071.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2024-02-20T12:46:50.029 INFO:teuthology.orchestra.run.smithi071.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2024-02-20T12:46:50.029 INFO:teuthology.orchestra.run.smithi071.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2024-02-20T12:46:50.029 INFO:teuthology.orchestra.run.smithi071.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2024-02-20T12:46:50.029 INFO:teuthology.orchestra.run.smithi071.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2024-02-20T12:46:50.029 INFO:teuthology.orchestra.run.smithi071.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2024-02-20T12:46:50.029 INFO:teuthology.orchestra.run.smithi071.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2024-02-20T12:46:50.029 INFO:teuthology.orchestra.run.smithi071.stdout: python3-werkzeug noarch 0.12.2-4.el8 appstream 457 k 2024-02-20T12:46:50.029 INFO:teuthology.orchestra.run.smithi071.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2024-02-20T12:46:50.029 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-20T12:46:50.030 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction Summary 2024-02-20T12:46:50.030 INFO:teuthology.orchestra.run.smithi071.stdout:================================================================================ 2024-02-20T12:46:50.030 INFO:teuthology.orchestra.run.smithi071.stdout:Install 32 Packages 2024-02-20T12:46:50.030 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-20T12:46:50.030 INFO:teuthology.orchestra.run.smithi071.stdout:Total download size: 32 M 2024-02-20T12:46:50.030 INFO:teuthology.orchestra.run.smithi071.stdout:Installed size: 104 M 2024-02-20T12:46:50.031 INFO:teuthology.orchestra.run.smithi071.stdout:Downloading Packages: 2024-02-20T12:46:50.498 INFO:teuthology.orchestra.run.smithi071.stdout:(1/32): ceph-16.2.14-548.g81bd20d6.el8.x86_64.r 50 kB/s | 6.4 kB 00:00 2024-02-20T12:46:50.522 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: libcephsqlite-2:16.2.14-548.g81bd20d6.el8.x86_64 5/6 2024-02-20T12:46:50.699 INFO:teuthology.orchestra.run.smithi071.stdout:(2/32): ceph-mds-16.2.14-548.g81bd20d6.el8.x86_ 7.4 MB/s | 2.4 MB 00:00 2024-02-20T12:46:50.965 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-test-2:16.2.14-548.g81bd20d6.el8.x86_64 6/6 2024-02-20T12:46:51.296 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-test-2:16.2.14-548.g81bd20d6.el8.x86_64 6/6 2024-02-20T12:46:51.296 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-test-2:16.2.14-548.g81bd20d6.el8.x86_64 1/6 2024-02-20T12:46:51.296 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libcephsqlite-2:16.2.14-548.g81bd20d6.el8.x86_64 2/6 2024-02-20T12:46:51.296 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : jq-1.6-8.el8.x86_64 3/6 2024-02-20T12:46:51.296 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : oniguruma-6.8.2-3.el8.x86_64 4/6 2024-02-20T12:46:51.297 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 5/6 2024-02-20T12:46:51.482 INFO:teuthology.orchestra.run.smithi071.stdout:(3/32): ceph-mon-16.2.14-548.g81bd20d6.el8.x86_ 5.3 MB/s | 5.2 MB 00:00 2024-02-20T12:46:51.549 INFO:teuthology.orchestra.run.smithi071.stdout:(4/32): ceph-mgr-modules-core-16.2.14-548.g81bd 3.6 MB/s | 245 kB 00:00 2024-02-20T12:46:51.725 INFO:teuthology.orchestra.run.smithi071.stdout:(5/32): ceph-osd-16.2.14-548.g81bd20d6.el8.x86_ 18 MB/s | 19 MB 00:01 2024-02-20T12:46:51.725 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 6/6 2024-02-20T12:46:51.725 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-20T12:46:51.726 INFO:teuthology.orchestra.run.smithi064.stdout:Installed: 2024-02-20T12:46:51.726 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-test-2:16.2.14-548.g81bd20d6.el8.x86_64 2024-02-20T12:46:51.726 INFO:teuthology.orchestra.run.smithi064.stdout: jq-1.6-8.el8.x86_64 2024-02-20T12:46:51.726 INFO:teuthology.orchestra.run.smithi064.stdout: libcephsqlite-2:16.2.14-548.g81bd20d6.el8.x86_64 2024-02-20T12:46:51.726 INFO:teuthology.orchestra.run.smithi064.stdout: oniguruma-6.8.2-3.el8.x86_64 2024-02-20T12:46:51.726 INFO:teuthology.orchestra.run.smithi064.stdout: socat-1.7.4.1-1.el8.x86_64 2024-02-20T12:46:51.726 INFO:teuthology.orchestra.run.smithi064.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2024-02-20T12:46:51.726 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-20T12:46:51.726 INFO:teuthology.orchestra.run.smithi064.stdout:Complete! 2024-02-20T12:46:51.875 INFO:teuthology.orchestra.run.smithi071.stdout:(6/32): ceph-mgr-16.2.14-548.g81bd20d6.el8.x86_ 1.1 MB/s | 1.6 MB 00:01 2024-02-20T12:46:51.997 DEBUG:teuthology.orchestra.run.smithi064:> sudo yum -y install ceph 2024-02-20T12:46:52.017 INFO:teuthology.orchestra.run.smithi071.stdout:(7/32): python3-markupsafe-0.23-19.el8.x86_64.r 133 kB/s | 39 kB 00:00 2024-02-20T12:46:52.109 INFO:teuthology.orchestra.run.smithi071.stdout:(8/32): python3-mako-1.0.6-14.el8.noarch.rpm 281 kB/s | 157 kB 00:00 2024-02-20T12:46:52.192 INFO:teuthology.orchestra.run.smithi071.stdout:(9/32): python3-pytz-2017.2-11.el8.noarch.rpm 325 kB/s | 57 kB 00:00 2024-02-20T12:46:52.284 INFO:teuthology.orchestra.run.smithi071.stdout:(10/32): python3-cffi-1.11.5-6.el8.x86_64.rpm 2.7 MB/s | 249 kB 00:00 2024-02-20T12:46:52.309 INFO:teuthology.orchestra.run.smithi071.stdout:(11/32): python3-pyOpenSSL-19.0.0-1.el8.noarch. 238 kB/s | 103 kB 00:00 2024-02-20T12:46:52.335 INFO:teuthology.orchestra.run.smithi071.stdout:(12/32): python3-werkzeug-0.12.2-4.el8.noarch.r 2.0 MB/s | 457 kB 00:00 2024-02-20T12:46:52.360 INFO:teuthology.orchestra.run.smithi071.stdout:(13/32): python3-cryptography-3.2.1-7.el8.x86_6 8.3 MB/s | 641 kB 00:00 2024-02-20T12:46:52.427 INFO:teuthology.orchestra.run.smithi071.stdout:(14/32): python3-pycparser-2.14-14.el8.noarch.r 931 kB/s | 109 kB 00:00 2024-02-20T12:46:52.452 INFO:teuthology.orchestra.run.smithi071.stdout:(15/32): python3-bcrypt-3.1.6-2.el8.1.x86_64.rp 372 kB/s | 44 kB 00:00 2024-02-20T12:46:52.544 INFO:teuthology.orchestra.run.smithi071.stdout:(16/32): python3-beautifulsoup4-4.6.3-2.el8.1.n 1.0 MB/s | 185 kB 00:00 2024-02-20T12:46:52.578 INFO:teuthology.orchestra.run.smithi071.stdout:(17/32): python3-cherrypy-18.4.0-1.el8.noarch.r 3.0 MB/s | 384 kB 00:00 2024-02-20T12:46:52.603 INFO:teuthology.orchestra.run.smithi071.stdout:(18/32): python3-cheroot-8.5.2-1.el8.noarch.rpm 984 kB/s | 173 kB 00:00 2024-02-20T12:46:52.628 INFO:teuthology.orchestra.run.smithi071.stdout:(19/32): python3-jaraco-6.2-6.el8.noarch.rpm 133 kB/s | 11 kB 00:00 2024-02-20T12:46:52.653 INFO:teuthology.orchestra.run.smithi071.stdout:(20/32): python3-jaraco-functools-2.0-4.el8.noa 233 kB/s | 18 kB 00:00 2024-02-20T12:46:52.679 INFO:teuthology.orchestra.run.smithi071.stdout:(21/32): python3-logutils-0.3.5-11.el8.noarch.r 969 kB/s | 49 kB 00:00 2024-02-20T12:46:52.704 INFO:teuthology.orchestra.run.smithi071.stdout:(22/32): python3-more-itertools-7.2.0-3.el8.noa 778 kB/s | 59 kB 00:00 2024-02-20T12:46:52.729 INFO:teuthology.orchestra.run.smithi071.stdout:(23/32): python3-pecan-1.3.2-9.el8.noarch.rpm 3.7 MB/s | 283 kB 00:00 2024-02-20T12:46:52.755 INFO:teuthology.orchestra.run.smithi071.stdout:(24/32): python3-portend-2.6-1.el8.noarch.rpm 312 kB/s | 16 kB 00:00 2024-02-20T12:46:52.801 INFO:teuthology.orchestra.run.smithi071.stdout:(25/32): python3-simplegeneric-0.8.1-17.el8.noa 195 kB/s | 19 kB 00:00 2024-02-20T12:46:52.824 INFO:teuthology.orchestra.run.smithi064.stdout:Last metadata expiration check: 0:00:58 ago on Tue 20 Feb 2024 12:45:54 PM UTC. 2024-02-20T12:46:52.830 INFO:teuthology.orchestra.run.smithi071.stdout:(26/32): python3-singledispatch-3.4.0.3-18.el8. 238 kB/s | 24 kB 00:00 2024-02-20T12:46:52.856 INFO:teuthology.orchestra.run.smithi071.stdout:(27/32): python3-tempora-1.14.1-5.el8.noarch.rp 291 kB/s | 29 kB 00:00 2024-02-20T12:46:52.881 INFO:teuthology.orchestra.run.smithi071.stdout:(28/32): python3-trustme-0.6.0-4.el8.noarch.rpm 346 kB/s | 27 kB 00:00 2024-02-20T12:46:52.906 INFO:teuthology.orchestra.run.smithi071.stdout:(29/32): python3-waitress-1.4.3-1.el8.noarch.rp 3.3 MB/s | 255 kB 00:00 2024-02-20T12:46:52.931 INFO:teuthology.orchestra.run.smithi071.stdout:(30/32): python3-webob-1.8.5-1.el8.1.noarch.rpm 3.3 MB/s | 251 kB 00:00 2024-02-20T12:46:52.956 INFO:teuthology.orchestra.run.smithi071.stdout:(31/32): python3-zc-lockfile-2.0-2.el8.noarch.r 459 kB/s | 23 kB 00:00 2024-02-20T12:46:52.981 INFO:teuthology.orchestra.run.smithi071.stdout:(32/32): python3-webtest-2.0.33-1.el8.noarch.rp 851 kB/s | 85 kB 00:00 2024-02-20T12:46:52.983 INFO:teuthology.orchestra.run.smithi071.stdout:-------------------------------------------------------------------------------- 2024-02-20T12:46:52.983 INFO:teuthology.orchestra.run.smithi071.stdout:Total 11 MB/s | 32 MB 00:02 2024-02-20T12:46:53.064 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction check 2024-02-20T12:46:53.102 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction check succeeded. 2024-02-20T12:46:53.103 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction test 2024-02-20T12:46:53.424 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction test succeeded. 2024-02-20T12:46:53.428 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction 2024-02-20T12:46:54.488 INFO:teuthology.orchestra.run.smithi064.stdout:Dependencies resolved. 2024-02-20T12:46:54.491 INFO:teuthology.orchestra.run.smithi064.stdout:================================================================================ 2024-02-20T12:46:54.492 INFO:teuthology.orchestra.run.smithi064.stdout: Package Arch Version Repository Size 2024-02-20T12:46:54.492 INFO:teuthology.orchestra.run.smithi064.stdout:================================================================================ 2024-02-20T12:46:54.492 INFO:teuthology.orchestra.run.smithi064.stdout:Installing: 2024-02-20T12:46:54.492 INFO:teuthology.orchestra.run.smithi064.stdout: ceph x86_64 2:16.2.14-548.g81bd20d6.el8 ceph 6.4 k 2024-02-20T12:46:54.492 INFO:teuthology.orchestra.run.smithi064.stdout:Installing dependencies: 2024-02-20T12:46:54.492 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mds x86_64 2:16.2.14-548.g81bd20d6.el8 ceph 2.4 M 2024-02-20T12:46:54.492 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mgr x86_64 2:16.2.14-548.g81bd20d6.el8 ceph 1.6 M 2024-02-20T12:46:54.493 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mgr-modules-core noarch 2:16.2.14-548.g81bd20d6.el8 ceph-noarch 245 k 2024-02-20T12:46:54.493 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mon x86_64 2:16.2.14-548.g81bd20d6.el8 ceph 5.2 M 2024-02-20T12:46:54.493 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-osd x86_64 2:16.2.14-548.g81bd20d6.el8 ceph 19 M 2024-02-20T12:46:54.493 INFO:teuthology.orchestra.run.smithi064.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2024-02-20T12:46:54.493 INFO:teuthology.orchestra.run.smithi064.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2024-02-20T12:46:54.493 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cffi x86_64 1.11.5-6.el8 baseos 249 k 2024-02-20T12:46:54.493 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2024-02-20T12:46:54.493 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2024-02-20T12:46:54.493 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cryptography x86_64 3.2.1-7.el8 baseos 641 k 2024-02-20T12:46:54.493 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2024-02-20T12:46:54.493 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2024-02-20T12:46:54.493 INFO:teuthology.orchestra.run.smithi064.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2024-02-20T12:46:54.494 INFO:teuthology.orchestra.run.smithi064.stdout: python3-mako noarch 1.0.6-14.el8 appstream 157 k 2024-02-20T12:46:54.494 INFO:teuthology.orchestra.run.smithi064.stdout: python3-markupsafe x86_64 0.23-19.el8 appstream 39 k 2024-02-20T12:46:54.494 INFO:teuthology.orchestra.run.smithi064.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2024-02-20T12:46:54.494 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2024-02-20T12:46:54.494 INFO:teuthology.orchestra.run.smithi064.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2024-02-20T12:46:54.494 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 appstream 103 k 2024-02-20T12:46:54.494 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pycparser noarch 2.14-14.el8 baseos 109 k 2024-02-20T12:46:54.494 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pytz noarch 2017.2-11.el8 appstream 57 k 2024-02-20T12:46:54.494 INFO:teuthology.orchestra.run.smithi064.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2024-02-20T12:46:54.494 INFO:teuthology.orchestra.run.smithi064.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2024-02-20T12:46:54.494 INFO:teuthology.orchestra.run.smithi064.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2024-02-20T12:46:54.494 INFO:teuthology.orchestra.run.smithi064.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2024-02-20T12:46:54.494 INFO:teuthology.orchestra.run.smithi064.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2024-02-20T12:46:54.494 INFO:teuthology.orchestra.run.smithi064.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2024-02-20T12:46:54.495 INFO:teuthology.orchestra.run.smithi064.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2024-02-20T12:46:54.495 INFO:teuthology.orchestra.run.smithi064.stdout: python3-werkzeug noarch 0.12.2-4.el8 appstream 457 k 2024-02-20T12:46:54.495 INFO:teuthology.orchestra.run.smithi064.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2024-02-20T12:46:54.495 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-20T12:46:54.495 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction Summary 2024-02-20T12:46:54.495 INFO:teuthology.orchestra.run.smithi064.stdout:================================================================================ 2024-02-20T12:46:54.495 INFO:teuthology.orchestra.run.smithi064.stdout:Install 32 Packages 2024-02-20T12:46:54.495 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-20T12:46:54.496 INFO:teuthology.orchestra.run.smithi064.stdout:Total download size: 32 M 2024-02-20T12:46:54.496 INFO:teuthology.orchestra.run.smithi064.stdout:Installed size: 104 M 2024-02-20T12:46:54.496 INFO:teuthology.orchestra.run.smithi064.stdout:Downloading Packages: 2024-02-20T12:46:54.567 INFO:teuthology.orchestra.run.smithi071.stdout: Preparing : 1/1 2024-02-20T12:46:54.826 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 1/32 2024-02-20T12:46:54.908 INFO:teuthology.orchestra.run.smithi064.stdout:(1/32): ceph-16.2.14-548.g81bd20d6.el8.x86_64.r 47 kB/s | 6.4 kB 00:00 2024-02-20T12:46:55.036 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 2/32 2024-02-20T12:46:55.092 INFO:teuthology.orchestra.run.smithi064.stdout:(2/32): ceph-mgr-16.2.14-548.g81bd20d6.el8.x86_ 5.1 MB/s | 1.6 MB 00:00 2024-02-20T12:46:55.344 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 3/32 2024-02-20T12:46:55.492 INFO:teuthology.orchestra.run.smithi064.stdout:(3/32): ceph-mon-16.2.14-548.g81bd20d6.el8.x86_ 8.9 MB/s | 5.2 MB 00:00 2024-02-20T12:46:55.559 INFO:teuthology.orchestra.run.smithi064.stdout:(4/32): ceph-mgr-modules-core-16.2.14-548.g81bd 3.6 MB/s | 245 kB 00:00 2024-02-20T12:46:55.601 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 4/32 2024-02-20T12:46:55.610 INFO:teuthology.orchestra.run.smithi064.stdout:(5/32): ceph-mds-16.2.14-548.g81bd20d6.el8.x86_ 2.9 MB/s | 2.4 MB 00:00 2024-02-20T12:46:55.815 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 5/32 2024-02-20T12:46:55.902 INFO:teuthology.orchestra.run.smithi064.stdout:(6/32): python3-markupsafe-0.23-19.el8.x86_64.r 133 kB/s | 39 kB 00:00 2024-02-20T12:46:55.994 INFO:teuthology.orchestra.run.smithi064.stdout:(7/32): python3-mako-1.0.6-14.el8.noarch.rpm 362 kB/s | 157 kB 00:00 2024-02-20T12:46:56.012 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 6/32 2024-02-20T12:46:56.205 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 7/32 2024-02-20T12:46:56.227 INFO:teuthology.orchestra.run.smithi064.stdout:(8/32): ceph-osd-16.2.14-548.g81bd20d6.el8.x86_ 16 MB/s | 19 MB 00:01 2024-02-20T12:46:56.257 INFO:teuthology.orchestra.run.smithi064.stdout:(9/32): python3-pyOpenSSL-19.0.0-1.el8.noarch.r 295 kB/s | 103 kB 00:00 2024-02-20T12:46:56.305 INFO:teuthology.orchestra.run.smithi064.stdout:(10/32): python3-pytz-2017.2-11.el8.noarch.rpm 183 kB/s | 57 kB 00:00 2024-02-20T12:46:56.374 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 8/32 2024-02-20T12:46:56.403 INFO:teuthology.orchestra.run.smithi064.stdout:(11/32): python3-cffi-1.11.5-6.el8.x86_64.rpm 2.5 MB/s | 249 kB 00:00 2024-02-20T12:46:56.428 INFO:teuthology.orchestra.run.smithi064.stdout:(12/32): python3-cryptography-3.2.1-7.el8.x86_6 5.1 MB/s | 641 kB 00:00 2024-02-20T12:46:56.454 INFO:teuthology.orchestra.run.smithi064.stdout:(13/32): python3-pycparser-2.14-14.el8.noarch.r 2.1 MB/s | 109 kB 00:00 2024-02-20T12:46:56.606 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 9/32 2024-02-20T12:46:56.754 INFO:teuthology.orchestra.run.smithi064.stdout:(14/32): python3-bcrypt-3.1.6-2.el8.1.x86_64.rp 134 kB/s | 44 kB 00:00 2024-02-20T12:46:56.821 INFO:teuthology.orchestra.run.smithi064.stdout:(15/32): python3-werkzeug-0.12.2-4.el8.noarch.r 771 kB/s | 457 kB 00:00 2024-02-20T12:46:56.835 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 10/32 2024-02-20T12:46:56.846 INFO:teuthology.orchestra.run.smithi064.stdout:(16/32): python3-beautifulsoup4-4.6.3-2.el8.1.n 472 kB/s | 185 kB 00:00 2024-02-20T12:46:56.888 INFO:teuthology.orchestra.run.smithi064.stdout:(17/32): python3-cheroot-8.5.2-1.el8.noarch.rpm 1.3 MB/s | 173 kB 00:00 2024-02-20T12:46:57.005 INFO:teuthology.orchestra.run.smithi064.stdout:(18/32): python3-jaraco-6.2-6.el8.noarch.rpm 70 kB/s | 11 kB 00:00 2024-02-20T12:46:57.030 INFO:teuthology.orchestra.run.smithi064.stdout:(19/32): python3-jaraco-functools-2.0-4.el8.noa 124 kB/s | 18 kB 00:00 2024-02-20T12:46:57.064 INFO:teuthology.orchestra.run.smithi064.stdout:(20/32): python3-cherrypy-18.4.0-1.el8.noarch.r 1.5 MB/s | 384 kB 00:00 2024-02-20T12:46:57.106 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 11/32 2024-02-20T12:46:57.136 INFO:teuthology.orchestra.run.smithi064.stdout:(21/32): python3-more-itertools-7.2.0-3.el8.noa 558 kB/s | 59 kB 00:00 2024-02-20T12:46:57.164 INFO:teuthology.orchestra.run.smithi064.stdout:(22/32): python3-logutils-0.3.5-11.el8.noarch.r 306 kB/s | 49 kB 00:00 2024-02-20T12:46:57.190 INFO:teuthology.orchestra.run.smithi064.stdout:(23/32): python3-pecan-1.3.2-9.el8.noarch.rpm 2.2 MB/s | 283 kB 00:00 2024-02-20T12:46:57.207 INFO:teuthology.orchestra.run.smithi064.stdout:(24/32): python3-portend-2.6-1.el8.noarch.rpm 221 kB/s | 16 kB 00:00 2024-02-20T12:46:57.232 INFO:teuthology.orchestra.run.smithi064.stdout:(25/32): python3-simplegeneric-0.8.1-17.el8.noa 278 kB/s | 19 kB 00:00 2024-02-20T12:46:57.258 INFO:teuthology.orchestra.run.smithi064.stdout:(26/32): python3-singledispatch-3.4.0.3-18.el8. 355 kB/s | 24 kB 00:00 2024-02-20T12:46:57.283 INFO:teuthology.orchestra.run.smithi064.stdout:(27/32): python3-tempora-1.14.1-5.el8.noarch.rp 387 kB/s | 29 kB 00:00 2024-02-20T12:46:57.357 INFO:teuthology.orchestra.run.smithi064.stdout:(28/32): python3-trustme-0.6.0-4.el8.noarch.rpm 222 kB/s | 27 kB 00:00 2024-02-20T12:46:57.380 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 12/32 2024-02-20T12:46:57.392 INFO:teuthology.orchestra.run.smithi064.stdout:(29/32): python3-webob-1.8.5-1.el8.1.noarch.rpm 2.3 MB/s | 251 kB 00:00 2024-02-20T12:46:57.417 INFO:teuthology.orchestra.run.smithi064.stdout:(30/32): python3-waitress-1.4.3-1.el8.noarch.rp 1.6 MB/s | 255 kB 00:00 2024-02-20T12:46:57.484 INFO:teuthology.orchestra.run.smithi064.stdout:(31/32): python3-zc-lockfile-2.0-2.el8.noarch.r 250 kB/s | 23 kB 00:00 2024-02-20T12:46:57.509 INFO:teuthology.orchestra.run.smithi064.stdout:(32/32): python3-webtest-2.0.33-1.el8.noarch.rp 561 kB/s | 85 kB 00:00 2024-02-20T12:46:57.510 INFO:teuthology.orchestra.run.smithi064.stdout:-------------------------------------------------------------------------------- 2024-02-20T12:46:57.511 INFO:teuthology.orchestra.run.smithi064.stdout:Total 11 MB/s | 32 MB 00:03 2024-02-20T12:46:57.596 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction check 2024-02-20T12:46:57.638 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction check succeeded. 2024-02-20T12:46:57.638 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction test 2024-02-20T12:46:57.768 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-cffi-1.11.5-6.el8.x86_64 13/32 2024-02-20T12:46:57.969 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction test succeeded. 2024-02-20T12:46:57.972 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction 2024-02-20T12:46:58.030 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-cryptography-3.2.1-7.el8.x86_64 14/32 2024-02-20T12:46:58.243 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 15/32 2024-02-20T12:46:58.518 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 16/32 2024-02-20T12:46:58.728 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 17/32 2024-02-20T12:46:59.062 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 18/32 2024-02-20T12:46:59.298 INFO:teuthology.orchestra.run.smithi064.stdout: Preparing : 1/1 2024-02-20T12:46:59.341 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 19/32 2024-02-20T12:46:59.529 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-pytz-2017.2-11.el8.noarch 20/32 2024-02-20T12:46:59.566 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 1/32 2024-02-20T12:46:59.757 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 21/32 2024-02-20T12:46:59.941 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 2/32 2024-02-20T12:46:59.990 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-portend-2.6-1.el8.noarch 22/32 2024-02-20T12:47:00.247 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 3/32 2024-02-20T12:47:00.255 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 23/32 2024-02-20T12:47:00.480 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 24/32 2024-02-20T12:47:00.538 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 4/32 2024-02-20T12:47:00.778 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 5/32 2024-02-20T12:47:00.800 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-mako-1.0.6-14.el8.noarch 25/32 2024-02-20T12:47:01.011 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 6/32 2024-02-20T12:47:01.139 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 26/32 2024-02-20T12:47:01.251 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 7/32 2024-02-20T12:47:01.488 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 8/32 2024-02-20T12:47:01.527 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-mgr-modules-core-2:16.2.14-548.g81bd20d6.el8. 27/32 2024-02-20T12:47:01.774 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-mgr-2:16.2.14-548.g81bd20d6.el8.x86_64 28/32 2024-02-20T12:47:01.807 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 9/32 2024-02-20T12:47:02.078 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 10/32 2024-02-20T12:47:02.366 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 11/32 2024-02-20T12:47:02.632 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 12/32 2024-02-20T12:47:03.084 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-cffi-1.11.5-6.el8.x86_64 13/32 2024-02-20T12:47:03.287 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: ceph-mgr-2:16.2.14-548.g81bd20d6.el8.x86_64 28/32 2024-02-20T12:47:03.360 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-cryptography-3.2.1-7.el8.x86_64 14/32 2024-02-20T12:47:03.510 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-osd-2:16.2.14-548.g81bd20d6.el8.x86_64 29/32 2024-02-20T12:47:03.639 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 15/32 2024-02-20T12:47:03.952 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: ceph-osd-2:16.2.14-548.g81bd20d6.el8.x86_64 29/32 2024-02-20T12:47:03.956 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 16/32 2024-02-20T12:47:04.185 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-mon-2:16.2.14-548.g81bd20d6.el8.x86_64 30/32 2024-02-20T12:47:04.251 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 17/32 2024-02-20T12:47:04.402 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: ceph-mon-2:16.2.14-548.g81bd20d6.el8.x86_64 30/32 2024-02-20T12:47:04.593 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-mds-2:16.2.14-548.g81bd20d6.el8.x86_64 31/32 2024-02-20T12:47:04.610 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: ceph-mds-2:16.2.14-548.g81bd20d6.el8.x86_64 31/32 2024-02-20T12:47:04.619 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 18/32 2024-02-20T12:47:04.734 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-2:16.2.14-548.g81bd20d6.el8.x86_64 32/32 2024-02-20T12:47:04.872 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 19/32 2024-02-20T12:47:05.069 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-pytz-2017.2-11.el8.noarch 20/32 2024-02-20T12:47:05.139 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: ceph-2:16.2.14-548.g81bd20d6.el8.x86_64 32/32 2024-02-20T12:47:05.139 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-2:16.2.14-548.g81bd20d6.el8.x86_64 1/32 2024-02-20T12:47:05.139 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-mds-2:16.2.14-548.g81bd20d6.el8.x86_64 2/32 2024-02-20T12:47:05.139 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-mgr-2:16.2.14-548.g81bd20d6.el8.x86_64 3/32 2024-02-20T12:47:05.139 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-mon-2:16.2.14-548.g81bd20d6.el8.x86_64 4/32 2024-02-20T12:47:05.140 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-osd-2:16.2.14-548.g81bd20d6.el8.x86_64 5/32 2024-02-20T12:47:05.140 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-mgr-modules-core-2:16.2.14-548.g81bd20d6.el8. 6/32 2024-02-20T12:47:05.140 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-mako-1.0.6-14.el8.noarch 7/32 2024-02-20T12:47:05.140 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 8/32 2024-02-20T12:47:05.140 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 9/32 2024-02-20T12:47:05.140 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-pytz-2017.2-11.el8.noarch 10/32 2024-02-20T12:47:05.140 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 11/32 2024-02-20T12:47:05.140 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-cffi-1.11.5-6.el8.x86_64 12/32 2024-02-20T12:47:05.140 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-cryptography-3.2.1-7.el8.x86_64 13/32 2024-02-20T12:47:05.140 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 14/32 2024-02-20T12:47:05.140 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 15/32 2024-02-20T12:47:05.140 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 16/32 2024-02-20T12:47:05.140 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 17/32 2024-02-20T12:47:05.141 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 18/32 2024-02-20T12:47:05.141 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 19/32 2024-02-20T12:47:05.141 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 20/32 2024-02-20T12:47:05.141 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 21/32 2024-02-20T12:47:05.141 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 22/32 2024-02-20T12:47:05.141 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 23/32 2024-02-20T12:47:05.141 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-portend-2.6-1.el8.noarch 24/32 2024-02-20T12:47:05.141 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 25/32 2024-02-20T12:47:05.141 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 26/32 2024-02-20T12:47:05.141 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 27/32 2024-02-20T12:47:05.142 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 28/32 2024-02-20T12:47:05.142 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 29/32 2024-02-20T12:47:05.142 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 30/32 2024-02-20T12:47:05.142 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 31/32 2024-02-20T12:47:05.287 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 21/32 2024-02-20T12:47:05.590 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-portend-2.6-1.el8.noarch 22/32 2024-02-20T12:47:05.952 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 23/32 2024-02-20T12:47:05.971 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 32/32 2024-02-20T12:47:05.971 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-20T12:47:05.971 INFO:teuthology.orchestra.run.smithi071.stdout:Installed: 2024-02-20T12:47:05.971 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-2:16.2.14-548.g81bd20d6.el8.x86_64 2024-02-20T12:47:05.972 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-mds-2:16.2.14-548.g81bd20d6.el8.x86_64 2024-02-20T12:47:05.972 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-mgr-2:16.2.14-548.g81bd20d6.el8.x86_64 2024-02-20T12:47:05.972 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-mgr-modules-core-2:16.2.14-548.g81bd20d6.el8.noarch 2024-02-20T12:47:05.972 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-mon-2:16.2.14-548.g81bd20d6.el8.x86_64 2024-02-20T12:47:05.972 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-osd-2:16.2.14-548.g81bd20d6.el8.x86_64 2024-02-20T12:47:05.972 INFO:teuthology.orchestra.run.smithi071.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2024-02-20T12:47:05.972 INFO:teuthology.orchestra.run.smithi071.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2024-02-20T12:47:05.972 INFO:teuthology.orchestra.run.smithi071.stdout: python3-cffi-1.11.5-6.el8.x86_64 2024-02-20T12:47:05.972 INFO:teuthology.orchestra.run.smithi071.stdout: python3-cheroot-8.5.2-1.el8.noarch 2024-02-20T12:47:05.972 INFO:teuthology.orchestra.run.smithi071.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2024-02-20T12:47:05.972 INFO:teuthology.orchestra.run.smithi071.stdout: python3-cryptography-3.2.1-7.el8.x86_64 2024-02-20T12:47:05.972 INFO:teuthology.orchestra.run.smithi071.stdout: python3-jaraco-6.2-6.el8.noarch 2024-02-20T12:47:05.972 INFO:teuthology.orchestra.run.smithi071.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2024-02-20T12:47:05.973 INFO:teuthology.orchestra.run.smithi071.stdout: python3-logutils-0.3.5-11.el8.noarch 2024-02-20T12:47:05.973 INFO:teuthology.orchestra.run.smithi071.stdout: python3-mako-1.0.6-14.el8.noarch 2024-02-20T12:47:05.973 INFO:teuthology.orchestra.run.smithi071.stdout: python3-markupsafe-0.23-19.el8.x86_64 2024-02-20T12:47:05.973 INFO:teuthology.orchestra.run.smithi071.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2024-02-20T12:47:05.973 INFO:teuthology.orchestra.run.smithi071.stdout: python3-pecan-1.3.2-9.el8.noarch 2024-02-20T12:47:05.973 INFO:teuthology.orchestra.run.smithi071.stdout: python3-portend-2.6-1.el8.noarch 2024-02-20T12:47:05.973 INFO:teuthology.orchestra.run.smithi071.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2024-02-20T12:47:05.973 INFO:teuthology.orchestra.run.smithi071.stdout: python3-pycparser-2.14-14.el8.noarch 2024-02-20T12:47:05.973 INFO:teuthology.orchestra.run.smithi071.stdout: python3-pytz-2017.2-11.el8.noarch 2024-02-20T12:47:05.973 INFO:teuthology.orchestra.run.smithi071.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2024-02-20T12:47:05.973 INFO:teuthology.orchestra.run.smithi071.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2024-02-20T12:47:05.973 INFO:teuthology.orchestra.run.smithi071.stdout: python3-tempora-1.14.1-5.el8.noarch 2024-02-20T12:47:05.973 INFO:teuthology.orchestra.run.smithi071.stdout: python3-trustme-0.6.0-4.el8.noarch 2024-02-20T12:47:05.973 INFO:teuthology.orchestra.run.smithi071.stdout: python3-waitress-1.4.3-1.el8.noarch 2024-02-20T12:47:05.973 INFO:teuthology.orchestra.run.smithi071.stdout: python3-webob-1.8.5-1.el8.1.noarch 2024-02-20T12:47:05.974 INFO:teuthology.orchestra.run.smithi071.stdout: python3-webtest-2.0.33-1.el8.noarch 2024-02-20T12:47:05.974 INFO:teuthology.orchestra.run.smithi071.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2024-02-20T12:47:05.974 INFO:teuthology.orchestra.run.smithi071.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2024-02-20T12:47:05.974 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-20T12:47:05.974 INFO:teuthology.orchestra.run.smithi071.stdout:Complete! 2024-02-20T12:47:06.185 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 24/32 2024-02-20T12:47:06.258 DEBUG:teuthology.orchestra.run.smithi071:> sudo yum -y install ceph-base 2024-02-20T12:47:06.550 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-mako-1.0.6-14.el8.noarch 25/32 2024-02-20T12:47:06.916 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 26/32 2024-02-20T12:47:07.018 INFO:teuthology.orchestra.run.smithi071.stdout:Last metadata expiration check: 0:01:16 ago on Tue 20 Feb 2024 12:45:51 PM UTC. 2024-02-20T12:47:07.325 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-mgr-modules-core-2:16.2.14-548.g81bd20d6.el8. 27/32 2024-02-20T12:47:07.522 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-mgr-2:16.2.14-548.g81bd20d6.el8.x86_64 28/32 2024-02-20T12:47:08.533 INFO:teuthology.orchestra.run.smithi071.stdout:Package ceph-base-2:16.2.14-548.g81bd20d6.el8.x86_64 is already installed. 2024-02-20T12:47:08.580 INFO:teuthology.orchestra.run.smithi071.stdout:Dependencies resolved. 2024-02-20T12:47:08.581 INFO:teuthology.orchestra.run.smithi071.stdout:Nothing to do. 2024-02-20T12:47:08.581 INFO:teuthology.orchestra.run.smithi071.stdout:Complete! 2024-02-20T12:47:08.693 DEBUG:teuthology.orchestra.run.smithi071:> sudo yum -y install cephadm 2024-02-20T12:47:09.171 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-mgr-2:16.2.14-548.g81bd20d6.el8.x86_64 28/32 2024-02-20T12:47:09.453 INFO:teuthology.orchestra.run.smithi071.stdout:Last metadata expiration check: 0:01:18 ago on Tue 20 Feb 2024 12:45:51 PM UTC. 2024-02-20T12:47:09.606 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-osd-2:16.2.14-548.g81bd20d6.el8.x86_64 29/32 2024-02-20T12:47:10.093 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-osd-2:16.2.14-548.g81bd20d6.el8.x86_64 29/32 2024-02-20T12:47:10.339 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-mon-2:16.2.14-548.g81bd20d6.el8.x86_64 30/32 2024-02-20T12:47:10.567 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-mon-2:16.2.14-548.g81bd20d6.el8.x86_64 30/32 2024-02-20T12:47:10.763 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-mds-2:16.2.14-548.g81bd20d6.el8.x86_64 31/32 2024-02-20T12:47:10.782 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-mds-2:16.2.14-548.g81bd20d6.el8.x86_64 31/32 2024-02-20T12:47:10.955 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-2:16.2.14-548.g81bd20d6.el8.x86_64 32/32 2024-02-20T12:47:11.005 INFO:teuthology.orchestra.run.smithi071.stdout:Dependencies resolved. 2024-02-20T12:47:11.006 INFO:teuthology.orchestra.run.smithi071.stdout:================================================================================ 2024-02-20T12:47:11.006 INFO:teuthology.orchestra.run.smithi071.stdout: Package Arch Version Repository Size 2024-02-20T12:47:11.006 INFO:teuthology.orchestra.run.smithi071.stdout:================================================================================ 2024-02-20T12:47:11.006 INFO:teuthology.orchestra.run.smithi071.stdout:Installing: 2024-02-20T12:47:11.006 INFO:teuthology.orchestra.run.smithi071.stdout: cephadm noarch 2:16.2.14-548.g81bd20d6.el8 ceph-noarch 85 k 2024-02-20T12:47:11.006 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-20T12:47:11.006 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction Summary 2024-02-20T12:47:11.006 INFO:teuthology.orchestra.run.smithi071.stdout:================================================================================ 2024-02-20T12:47:11.007 INFO:teuthology.orchestra.run.smithi071.stdout:Install 1 Package 2024-02-20T12:47:11.007 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-20T12:47:11.007 INFO:teuthology.orchestra.run.smithi071.stdout:Total download size: 85 k 2024-02-20T12:47:11.007 INFO:teuthology.orchestra.run.smithi071.stdout:Installed size: 352 k 2024-02-20T12:47:11.007 INFO:teuthology.orchestra.run.smithi071.stdout:Downloading Packages: 2024-02-20T12:47:11.170 INFO:teuthology.orchestra.run.smithi071.stdout:cephadm-16.2.14-548.g81bd20d6.el8.noarch.rpm 523 kB/s | 85 kB 00:00 2024-02-20T12:47:11.170 INFO:teuthology.orchestra.run.smithi071.stdout:-------------------------------------------------------------------------------- 2024-02-20T12:47:11.171 INFO:teuthology.orchestra.run.smithi071.stdout:Total 520 kB/s | 85 kB 00:00 2024-02-20T12:47:11.171 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction check 2024-02-20T12:47:11.177 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction check succeeded. 2024-02-20T12:47:11.178 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction test 2024-02-20T12:47:11.179 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction test succeeded. 2024-02-20T12:47:11.182 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction 2024-02-20T12:47:11.228 INFO:teuthology.orchestra.run.smithi071.stdout: Preparing : 1/1 2024-02-20T12:47:11.390 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-2:16.2.14-548.g81bd20d6.el8.x86_64 32/32 2024-02-20T12:47:11.390 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-2:16.2.14-548.g81bd20d6.el8.x86_64 1/32 2024-02-20T12:47:11.390 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-mds-2:16.2.14-548.g81bd20d6.el8.x86_64 2/32 2024-02-20T12:47:11.390 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-mgr-2:16.2.14-548.g81bd20d6.el8.x86_64 3/32 2024-02-20T12:47:11.390 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-mon-2:16.2.14-548.g81bd20d6.el8.x86_64 4/32 2024-02-20T12:47:11.390 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-osd-2:16.2.14-548.g81bd20d6.el8.x86_64 5/32 2024-02-20T12:47:11.390 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-mgr-modules-core-2:16.2.14-548.g81bd20d6.el8. 6/32 2024-02-20T12:47:11.390 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-mako-1.0.6-14.el8.noarch 7/32 2024-02-20T12:47:11.390 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 8/32 2024-02-20T12:47:11.390 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 9/32 2024-02-20T12:47:11.390 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-pytz-2017.2-11.el8.noarch 10/32 2024-02-20T12:47:11.390 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 11/32 2024-02-20T12:47:11.391 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-cffi-1.11.5-6.el8.x86_64 12/32 2024-02-20T12:47:11.391 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-cryptography-3.2.1-7.el8.x86_64 13/32 2024-02-20T12:47:11.391 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 14/32 2024-02-20T12:47:11.391 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 15/32 2024-02-20T12:47:11.391 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 16/32 2024-02-20T12:47:11.391 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 17/32 2024-02-20T12:47:11.391 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 18/32 2024-02-20T12:47:11.391 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 19/32 2024-02-20T12:47:11.391 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 20/32 2024-02-20T12:47:11.391 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 21/32 2024-02-20T12:47:11.391 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 22/32 2024-02-20T12:47:11.391 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 23/32 2024-02-20T12:47:11.391 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-portend-2.6-1.el8.noarch 24/32 2024-02-20T12:47:11.391 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 25/32 2024-02-20T12:47:11.391 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 26/32 2024-02-20T12:47:11.391 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 27/32 2024-02-20T12:47:11.392 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 28/32 2024-02-20T12:47:11.392 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 29/32 2024-02-20T12:47:11.392 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 30/32 2024-02-20T12:47:11.392 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 31/32 2024-02-20T12:47:11.711 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: cephadm-2:16.2.14-548.g81bd20d6.el8.noarch 1/1 2024-02-20T12:47:11.901 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : cephadm-2:16.2.14-548.g81bd20d6.el8.noarch 1/1 2024-02-20T12:47:12.134 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: cephadm-2:16.2.14-548.g81bd20d6.el8.noarch 1/1 2024-02-20T12:47:12.377 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 32/32 2024-02-20T12:47:12.377 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-20T12:47:12.377 INFO:teuthology.orchestra.run.smithi064.stdout:Installed: 2024-02-20T12:47:12.377 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-2:16.2.14-548.g81bd20d6.el8.x86_64 2024-02-20T12:47:12.377 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mds-2:16.2.14-548.g81bd20d6.el8.x86_64 2024-02-20T12:47:12.378 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mgr-2:16.2.14-548.g81bd20d6.el8.x86_64 2024-02-20T12:47:12.378 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mgr-modules-core-2:16.2.14-548.g81bd20d6.el8.noarch 2024-02-20T12:47:12.378 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mon-2:16.2.14-548.g81bd20d6.el8.x86_64 2024-02-20T12:47:12.378 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-osd-2:16.2.14-548.g81bd20d6.el8.x86_64 2024-02-20T12:47:12.378 INFO:teuthology.orchestra.run.smithi064.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2024-02-20T12:47:12.378 INFO:teuthology.orchestra.run.smithi064.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2024-02-20T12:47:12.378 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cffi-1.11.5-6.el8.x86_64 2024-02-20T12:47:12.378 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cheroot-8.5.2-1.el8.noarch 2024-02-20T12:47:12.378 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2024-02-20T12:47:12.378 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cryptography-3.2.1-7.el8.x86_64 2024-02-20T12:47:12.378 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jaraco-6.2-6.el8.noarch 2024-02-20T12:47:12.378 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2024-02-20T12:47:12.378 INFO:teuthology.orchestra.run.smithi064.stdout: python3-logutils-0.3.5-11.el8.noarch 2024-02-20T12:47:12.378 INFO:teuthology.orchestra.run.smithi064.stdout: python3-mako-1.0.6-14.el8.noarch 2024-02-20T12:47:12.378 INFO:teuthology.orchestra.run.smithi064.stdout: python3-markupsafe-0.23-19.el8.x86_64 2024-02-20T12:47:12.378 INFO:teuthology.orchestra.run.smithi064.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2024-02-20T12:47:12.379 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pecan-1.3.2-9.el8.noarch 2024-02-20T12:47:12.379 INFO:teuthology.orchestra.run.smithi064.stdout: python3-portend-2.6-1.el8.noarch 2024-02-20T12:47:12.379 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2024-02-20T12:47:12.379 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pycparser-2.14-14.el8.noarch 2024-02-20T12:47:12.379 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pytz-2017.2-11.el8.noarch 2024-02-20T12:47:12.379 INFO:teuthology.orchestra.run.smithi064.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2024-02-20T12:47:12.379 INFO:teuthology.orchestra.run.smithi064.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2024-02-20T12:47:12.379 INFO:teuthology.orchestra.run.smithi064.stdout: python3-tempora-1.14.1-5.el8.noarch 2024-02-20T12:47:12.379 INFO:teuthology.orchestra.run.smithi064.stdout: python3-trustme-0.6.0-4.el8.noarch 2024-02-20T12:47:12.379 INFO:teuthology.orchestra.run.smithi064.stdout: python3-waitress-1.4.3-1.el8.noarch 2024-02-20T12:47:12.379 INFO:teuthology.orchestra.run.smithi064.stdout: python3-webob-1.8.5-1.el8.1.noarch 2024-02-20T12:47:12.379 INFO:teuthology.orchestra.run.smithi064.stdout: python3-webtest-2.0.33-1.el8.noarch 2024-02-20T12:47:12.379 INFO:teuthology.orchestra.run.smithi064.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2024-02-20T12:47:12.379 INFO:teuthology.orchestra.run.smithi064.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2024-02-20T12:47:12.379 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-20T12:47:12.380 INFO:teuthology.orchestra.run.smithi064.stdout:Complete! 2024-02-20T12:47:12.653 DEBUG:teuthology.orchestra.run.smithi064:> sudo yum -y install ceph-base 2024-02-20T12:47:12.779 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : cephadm-2:16.2.14-548.g81bd20d6.el8.noarch 1/1 2024-02-20T12:47:12.779 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-20T12:47:12.779 INFO:teuthology.orchestra.run.smithi071.stdout:Installed: 2024-02-20T12:47:12.779 INFO:teuthology.orchestra.run.smithi071.stdout: cephadm-2:16.2.14-548.g81bd20d6.el8.noarch 2024-02-20T12:47:12.779 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-20T12:47:12.779 INFO:teuthology.orchestra.run.smithi071.stdout:Complete! 2024-02-20T12:47:12.976 DEBUG:teuthology.orchestra.run.smithi071:> sudo yum -y install ceph-immutable-object-cache 2024-02-20T12:47:13.444 INFO:teuthology.orchestra.run.smithi064.stdout:Last metadata expiration check: 0:01:19 ago on Tue 20 Feb 2024 12:45:54 PM UTC. 2024-02-20T12:47:13.741 INFO:teuthology.orchestra.run.smithi071.stdout:Last metadata expiration check: 0:01:22 ago on Tue 20 Feb 2024 12:45:51 PM UTC. 2024-02-20T12:47:15.052 INFO:teuthology.orchestra.run.smithi064.stdout:Package ceph-base-2:16.2.14-548.g81bd20d6.el8.x86_64 is already installed. 2024-02-20T12:47:15.102 INFO:teuthology.orchestra.run.smithi064.stdout:Dependencies resolved. 2024-02-20T12:47:15.104 INFO:teuthology.orchestra.run.smithi064.stdout:Nothing to do. 2024-02-20T12:47:15.104 INFO:teuthology.orchestra.run.smithi064.stdout:Complete! 2024-02-20T12:47:15.207 DEBUG:teuthology.orchestra.run.smithi064:> sudo yum -y install cephadm 2024-02-20T12:47:15.290 INFO:teuthology.orchestra.run.smithi071.stdout:Dependencies resolved. 2024-02-20T12:47:15.291 INFO:teuthology.orchestra.run.smithi071.stdout:================================================================================ 2024-02-20T12:47:15.291 INFO:teuthology.orchestra.run.smithi071.stdout: Package Arch Version Repo Size 2024-02-20T12:47:15.291 INFO:teuthology.orchestra.run.smithi071.stdout:================================================================================ 2024-02-20T12:47:15.291 INFO:teuthology.orchestra.run.smithi071.stdout:Installing: 2024-02-20T12:47:15.291 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-immutable-object-cache x86_64 2:16.2.14-548.g81bd20d6.el8 ceph 161 k 2024-02-20T12:47:15.292 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-20T12:47:15.292 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction Summary 2024-02-20T12:47:15.292 INFO:teuthology.orchestra.run.smithi071.stdout:================================================================================ 2024-02-20T12:47:15.292 INFO:teuthology.orchestra.run.smithi071.stdout:Install 1 Package 2024-02-20T12:47:15.292 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-20T12:47:15.292 INFO:teuthology.orchestra.run.smithi071.stdout:Total download size: 161 k 2024-02-20T12:47:15.292 INFO:teuthology.orchestra.run.smithi071.stdout:Installed size: 470 k 2024-02-20T12:47:15.292 INFO:teuthology.orchestra.run.smithi071.stdout:Downloading Packages: 2024-02-20T12:47:15.641 INFO:teuthology.orchestra.run.smithi071.stdout:ceph-immutable-object-cache-16.2.14-548.g81bd20 462 kB/s | 161 kB 00:00 2024-02-20T12:47:15.642 INFO:teuthology.orchestra.run.smithi071.stdout:-------------------------------------------------------------------------------- 2024-02-20T12:47:15.642 INFO:teuthology.orchestra.run.smithi071.stdout:Total 461 kB/s | 161 kB 00:00 2024-02-20T12:47:15.642 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction check 2024-02-20T12:47:15.652 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction check succeeded. 2024-02-20T12:47:15.653 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction test 2024-02-20T12:47:15.701 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction test succeeded. 2024-02-20T12:47:15.705 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction 2024-02-20T12:47:15.816 INFO:teuthology.orchestra.run.smithi071.stdout: Preparing : 1/1 2024-02-20T12:47:15.989 INFO:teuthology.orchestra.run.smithi064.stdout:Last metadata expiration check: 0:01:21 ago on Tue 20 Feb 2024 12:45:54 PM UTC. 2024-02-20T12:47:16.049 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-immutable-object-cache-2:16.2.14-548.g81bd20d6. 1/1 2024-02-20T12:47:16.461 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: ceph-immutable-object-cache-2:16.2.14-548.g81bd20d6. 1/1 2024-02-20T12:47:16.838 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-immutable-object-cache-2:16.2.14-548.g81bd20d6. 1/1 2024-02-20T12:47:16.838 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-20T12:47:16.838 INFO:teuthology.orchestra.run.smithi071.stdout:Installed: 2024-02-20T12:47:16.838 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-immutable-object-cache-2:16.2.14-548.g81bd20d6.el8.x86_64 2024-02-20T12:47:16.838 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-20T12:47:16.838 INFO:teuthology.orchestra.run.smithi071.stdout:Complete! 2024-02-20T12:47:17.044 DEBUG:teuthology.orchestra.run.smithi071:> sudo yum -y install ceph-mgr 2024-02-20T12:47:17.641 INFO:teuthology.orchestra.run.smithi064.stdout:Dependencies resolved. 2024-02-20T12:47:17.642 INFO:teuthology.orchestra.run.smithi064.stdout:================================================================================ 2024-02-20T12:47:17.642 INFO:teuthology.orchestra.run.smithi064.stdout: Package Arch Version Repository Size 2024-02-20T12:47:17.642 INFO:teuthology.orchestra.run.smithi064.stdout:================================================================================ 2024-02-20T12:47:17.642 INFO:teuthology.orchestra.run.smithi064.stdout:Installing: 2024-02-20T12:47:17.642 INFO:teuthology.orchestra.run.smithi064.stdout: cephadm noarch 2:16.2.14-548.g81bd20d6.el8 ceph-noarch 85 k 2024-02-20T12:47:17.642 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-20T12:47:17.642 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction Summary 2024-02-20T12:47:17.643 INFO:teuthology.orchestra.run.smithi064.stdout:================================================================================ 2024-02-20T12:47:17.643 INFO:teuthology.orchestra.run.smithi064.stdout:Install 1 Package 2024-02-20T12:47:17.643 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-20T12:47:17.643 INFO:teuthology.orchestra.run.smithi064.stdout:Total download size: 85 k 2024-02-20T12:47:17.643 INFO:teuthology.orchestra.run.smithi064.stdout:Installed size: 352 k 2024-02-20T12:47:17.643 INFO:teuthology.orchestra.run.smithi064.stdout:Downloading Packages: 2024-02-20T12:47:17.804 INFO:teuthology.orchestra.run.smithi071.stdout:Last metadata expiration check: 0:01:26 ago on Tue 20 Feb 2024 12:45:51 PM UTC. 2024-02-20T12:47:17.901 INFO:teuthology.orchestra.run.smithi064.stdout:cephadm-16.2.14-548.g81bd20d6.el8.noarch.rpm 331 kB/s | 85 kB 00:00 2024-02-20T12:47:17.901 INFO:teuthology.orchestra.run.smithi064.stdout:-------------------------------------------------------------------------------- 2024-02-20T12:47:17.902 INFO:teuthology.orchestra.run.smithi064.stdout:Total 329 kB/s | 85 kB 00:00 2024-02-20T12:47:17.902 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction check 2024-02-20T12:47:17.909 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction check succeeded. 2024-02-20T12:47:17.909 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction test 2024-02-20T12:47:17.911 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction test succeeded. 2024-02-20T12:47:17.915 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction 2024-02-20T12:47:17.959 INFO:teuthology.orchestra.run.smithi064.stdout: Preparing : 1/1 2024-02-20T12:47:18.551 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: cephadm-2:16.2.14-548.g81bd20d6.el8.noarch 1/1 2024-02-20T12:47:18.750 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : cephadm-2:16.2.14-548.g81bd20d6.el8.noarch 1/1 2024-02-20T12:47:18.992 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: cephadm-2:16.2.14-548.g81bd20d6.el8.noarch 1/1 2024-02-20T12:47:19.311 INFO:teuthology.orchestra.run.smithi071.stdout:Package ceph-mgr-2:16.2.14-548.g81bd20d6.el8.x86_64 is already installed. 2024-02-20T12:47:19.358 INFO:teuthology.orchestra.run.smithi071.stdout:Dependencies resolved. 2024-02-20T12:47:19.359 INFO:teuthology.orchestra.run.smithi071.stdout:Nothing to do. 2024-02-20T12:47:19.359 INFO:teuthology.orchestra.run.smithi071.stdout:Complete! 2024-02-20T12:47:19.403 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : cephadm-2:16.2.14-548.g81bd20d6.el8.noarch 1/1 2024-02-20T12:47:19.404 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-20T12:47:19.404 INFO:teuthology.orchestra.run.smithi064.stdout:Installed: 2024-02-20T12:47:19.404 INFO:teuthology.orchestra.run.smithi064.stdout: cephadm-2:16.2.14-548.g81bd20d6.el8.noarch 2024-02-20T12:47:19.404 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-20T12:47:19.404 INFO:teuthology.orchestra.run.smithi064.stdout:Complete! 2024-02-20T12:47:19.453 DEBUG:teuthology.orchestra.run.smithi071:> sudo yum -y install ceph-mgr-dashboard 2024-02-20T12:47:19.658 DEBUG:teuthology.orchestra.run.smithi064:> sudo yum -y install ceph-immutable-object-cache 2024-02-20T12:47:20.204 INFO:teuthology.orchestra.run.smithi071.stdout:Last metadata expiration check: 0:01:29 ago on Tue 20 Feb 2024 12:45:51 PM UTC. 2024-02-20T12:47:20.448 INFO:teuthology.orchestra.run.smithi064.stdout:Last metadata expiration check: 0:01:26 ago on Tue 20 Feb 2024 12:45:54 PM UTC. 2024-02-20T12:47:21.762 INFO:teuthology.orchestra.run.smithi071.stdout:Dependencies resolved. 2024-02-20T12:47:21.763 INFO:teuthology.orchestra.run.smithi071.stdout:================================================================================ 2024-02-20T12:47:21.763 INFO:teuthology.orchestra.run.smithi071.stdout: Package Arch Version Repository Size 2024-02-20T12:47:21.763 INFO:teuthology.orchestra.run.smithi071.stdout:================================================================================ 2024-02-20T12:47:21.763 INFO:teuthology.orchestra.run.smithi071.stdout:Installing: 2024-02-20T12:47:21.764 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-mgr-dashboard noarch 2:16.2.14-548.g81bd20d6.el8 ceph-noarch 3.1 M 2024-02-20T12:47:21.764 INFO:teuthology.orchestra.run.smithi071.stdout:Installing dependencies: 2024-02-20T12:47:21.764 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-grafana-dashboards noarch 2:16.2.14-548.g81bd20d6.el8 ceph-noarch 24 k 2024-02-20T12:47:21.764 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-prometheus-alerts noarch 2:16.2.14-548.g81bd20d6.el8 ceph-noarch 15 k 2024-02-20T12:47:21.764 INFO:teuthology.orchestra.run.smithi071.stdout: python3-jwt noarch 1.6.1-2.el8 baseos 43 k 2024-02-20T12:47:21.764 INFO:teuthology.orchestra.run.smithi071.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2024-02-20T12:47:21.764 INFO:teuthology.orchestra.run.smithi071.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2024-02-20T12:47:21.764 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-20T12:47:21.764 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction Summary 2024-02-20T12:47:21.764 INFO:teuthology.orchestra.run.smithi071.stdout:================================================================================ 2024-02-20T12:47:21.764 INFO:teuthology.orchestra.run.smithi071.stdout:Install 6 Packages 2024-02-20T12:47:21.764 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-20T12:47:21.765 INFO:teuthology.orchestra.run.smithi071.stdout:Total download size: 3.4 M 2024-02-20T12:47:21.765 INFO:teuthology.orchestra.run.smithi071.stdout:Installed size: 22 M 2024-02-20T12:47:21.765 INFO:teuthology.orchestra.run.smithi071.stdout:Downloading Packages: 2024-02-20T12:47:22.116 INFO:teuthology.orchestra.run.smithi064.stdout:Dependencies resolved. 2024-02-20T12:47:22.117 INFO:teuthology.orchestra.run.smithi064.stdout:================================================================================ 2024-02-20T12:47:22.117 INFO:teuthology.orchestra.run.smithi064.stdout: Package Arch Version Repo Size 2024-02-20T12:47:22.117 INFO:teuthology.orchestra.run.smithi064.stdout:================================================================================ 2024-02-20T12:47:22.117 INFO:teuthology.orchestra.run.smithi064.stdout:Installing: 2024-02-20T12:47:22.118 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-immutable-object-cache x86_64 2:16.2.14-548.g81bd20d6.el8 ceph 161 k 2024-02-20T12:47:22.118 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-20T12:47:22.118 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction Summary 2024-02-20T12:47:22.118 INFO:teuthology.orchestra.run.smithi064.stdout:================================================================================ 2024-02-20T12:47:22.118 INFO:teuthology.orchestra.run.smithi064.stdout:Install 1 Package 2024-02-20T12:47:22.118 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-20T12:47:22.118 INFO:teuthology.orchestra.run.smithi064.stdout:Total download size: 161 k 2024-02-20T12:47:22.118 INFO:teuthology.orchestra.run.smithi064.stdout:Installed size: 470 k 2024-02-20T12:47:22.119 INFO:teuthology.orchestra.run.smithi064.stdout:Downloading Packages: 2024-02-20T12:47:22.151 INFO:teuthology.orchestra.run.smithi071.stdout:(1/6): ceph-grafana-dashboards-16.2.14-548.g81b 174 kB/s | 24 kB 00:00 2024-02-20T12:47:22.260 INFO:teuthology.orchestra.run.smithi071.stdout:(2/6): ceph-prometheus-alerts-16.2.14-548.g81bd 60 kB/s | 15 kB 00:00 2024-02-20T12:47:22.352 INFO:teuthology.orchestra.run.smithi071.stdout:(3/6): python3-jwt-1.6.1-2.el8.noarch.rpm 213 kB/s | 43 kB 00:00 2024-02-20T12:47:22.508 INFO:teuthology.orchestra.run.smithi064.stdout:ceph-immutable-object-cache-16.2.14-548.g81bd20 413 kB/s | 161 kB 00:00 2024-02-20T12:47:22.508 INFO:teuthology.orchestra.run.smithi064.stdout:-------------------------------------------------------------------------------- 2024-02-20T12:47:22.509 INFO:teuthology.orchestra.run.smithi064.stdout:Total 412 kB/s | 161 kB 00:00 2024-02-20T12:47:22.509 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction check 2024-02-20T12:47:22.520 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction check succeeded. 2024-02-20T12:47:22.521 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction test 2024-02-20T12:47:22.544 INFO:teuthology.orchestra.run.smithi071.stdout:(4/6): python3-repoze-lru-0.7-6.el8.noarch.rpm 119 kB/s | 34 kB 00:00 2024-02-20T12:47:22.574 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction test succeeded. 2024-02-20T12:47:22.577 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction 2024-02-20T12:47:22.685 INFO:teuthology.orchestra.run.smithi064.stdout: Preparing : 1/1 2024-02-20T12:47:22.803 INFO:teuthology.orchestra.run.smithi071.stdout:(5/6): ceph-mgr-dashboard-16.2.14-548.g81bd20d6 4.0 MB/s | 3.1 MB 00:00 2024-02-20T12:47:22.853 INFO:teuthology.orchestra.run.smithi071.stdout:(6/6): python3-routes-2.4.1-12.el8.noarch.rpm 392 kB/s | 196 kB 00:00 2024-02-20T12:47:22.854 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-immutable-object-cache-2:16.2.14-548.g81bd20d6. 1/1 2024-02-20T12:47:22.855 INFO:teuthology.orchestra.run.smithi071.stdout:-------------------------------------------------------------------------------- 2024-02-20T12:47:22.855 INFO:teuthology.orchestra.run.smithi071.stdout:Total 3.1 MB/s | 3.4 MB 00:01 2024-02-20T12:47:22.866 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction check 2024-02-20T12:47:22.877 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction check succeeded. 2024-02-20T12:47:22.877 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction test 2024-02-20T12:47:23.038 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction test succeeded. 2024-02-20T12:47:23.042 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction 2024-02-20T12:47:23.288 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-immutable-object-cache-2:16.2.14-548.g81bd20d6. 1/1 2024-02-20T12:47:23.303 INFO:teuthology.orchestra.run.smithi071.stdout: Preparing : 1/1 2024-02-20T12:47:23.535 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 1/6 2024-02-20T12:47:23.704 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-immutable-object-cache-2:16.2.14-548.g81bd20d6. 1/1 2024-02-20T12:47:23.704 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-20T12:47:23.704 INFO:teuthology.orchestra.run.smithi064.stdout:Installed: 2024-02-20T12:47:23.705 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-immutable-object-cache-2:16.2.14-548.g81bd20d6.el8.x86_64 2024-02-20T12:47:23.705 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-20T12:47:23.705 INFO:teuthology.orchestra.run.smithi064.stdout:Complete! 2024-02-20T12:47:23.806 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 2/6 2024-02-20T12:47:23.948 DEBUG:teuthology.orchestra.run.smithi064:> sudo yum -y install ceph-mgr 2024-02-20T12:47:24.032 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 3/6 2024-02-20T12:47:24.178 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-prometheus-alerts-2:16.2.14-548.g81bd20d6.el8.n 4/6 2024-02-20T12:47:24.744 INFO:teuthology.orchestra.run.smithi064.stdout:Last metadata expiration check: 0:01:30 ago on Tue 20 Feb 2024 12:45:54 PM UTC. 2024-02-20T12:47:25.194 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-grafana-dashboards-2:16.2.14-548.g81bd20d6.el8. 5/6 2024-02-20T12:47:25.468 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-mgr-dashboard-2:16.2.14-548.g81bd20d6.el8.noarc 6/6 2024-02-20T12:47:25.750 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: ceph-mgr-dashboard-2:16.2.14-548.g81bd20d6.el8.noarc 6/6 2024-02-20T12:47:25.750 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-grafana-dashboards-2:16.2.14-548.g81bd20d6.el8. 1/6 2024-02-20T12:47:25.751 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-mgr-dashboard-2:16.2.14-548.g81bd20d6.el8.noarc 2/6 2024-02-20T12:47:25.751 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-prometheus-alerts-2:16.2.14-548.g81bd20d6.el8.n 3/6 2024-02-20T12:47:25.751 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 4/6 2024-02-20T12:47:25.751 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 5/6 2024-02-20T12:47:26.161 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 6/6 2024-02-20T12:47:26.161 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-20T12:47:26.161 INFO:teuthology.orchestra.run.smithi071.stdout:Installed: 2024-02-20T12:47:26.162 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-grafana-dashboards-2:16.2.14-548.g81bd20d6.el8.noarch 2024-02-20T12:47:26.162 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-mgr-dashboard-2:16.2.14-548.g81bd20d6.el8.noarch 2024-02-20T12:47:26.162 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-prometheus-alerts-2:16.2.14-548.g81bd20d6.el8.noarch 2024-02-20T12:47:26.162 INFO:teuthology.orchestra.run.smithi071.stdout: python3-jwt-1.6.1-2.el8.noarch 2024-02-20T12:47:26.162 INFO:teuthology.orchestra.run.smithi071.stdout: python3-repoze-lru-0.7-6.el8.noarch 2024-02-20T12:47:26.162 INFO:teuthology.orchestra.run.smithi071.stdout: python3-routes-2.4.1-12.el8.noarch 2024-02-20T12:47:26.162 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-20T12:47:26.162 INFO:teuthology.orchestra.run.smithi071.stdout:Complete! 2024-02-20T12:47:26.279 DEBUG:teuthology.orchestra.run.smithi071:> sudo yum -y install ceph-mgr-diskprediction-local 2024-02-20T12:47:26.355 INFO:teuthology.orchestra.run.smithi064.stdout:Package ceph-mgr-2:16.2.14-548.g81bd20d6.el8.x86_64 is already installed. 2024-02-20T12:47:26.406 INFO:teuthology.orchestra.run.smithi064.stdout:Dependencies resolved. 2024-02-20T12:47:26.407 INFO:teuthology.orchestra.run.smithi064.stdout:Nothing to do. 2024-02-20T12:47:26.407 INFO:teuthology.orchestra.run.smithi064.stdout:Complete! 2024-02-20T12:47:26.514 DEBUG:teuthology.orchestra.run.smithi064:> sudo yum -y install ceph-mgr-dashboard 2024-02-20T12:47:27.027 INFO:teuthology.orchestra.run.smithi071.stdout:Last metadata expiration check: 0:01:36 ago on Tue 20 Feb 2024 12:45:51 PM UTC. 2024-02-20T12:47:27.302 INFO:teuthology.orchestra.run.smithi064.stdout:Last metadata expiration check: 0:01:33 ago on Tue 20 Feb 2024 12:45:54 PM UTC. 2024-02-20T12:47:28.579 INFO:teuthology.orchestra.run.smithi071.stdout:Dependencies resolved. 2024-02-20T12:47:28.580 INFO:teuthology.orchestra.run.smithi071.stdout:============================================================================================ 2024-02-20T12:47:28.581 INFO:teuthology.orchestra.run.smithi071.stdout: Package Arch Version Repository Size 2024-02-20T12:47:28.581 INFO:teuthology.orchestra.run.smithi071.stdout:============================================================================================ 2024-02-20T12:47:28.581 INFO:teuthology.orchestra.run.smithi071.stdout:Installing: 2024-02-20T12:47:28.581 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-mgr-diskprediction-local noarch 2:16.2.14-548.g81bd20d6.el8 ceph-noarch 8.6 M 2024-02-20T12:47:28.581 INFO:teuthology.orchestra.run.smithi071.stdout:Installing dependencies: 2024-02-20T12:47:28.581 INFO:teuthology.orchestra.run.smithi071.stdout: libgfortran x86_64 8.5.0-21.el8 baseos 645 k 2024-02-20T12:47:28.581 INFO:teuthology.orchestra.run.smithi071.stdout: libquadmath x86_64 8.5.0-21.el8 baseos 172 k 2024-02-20T12:47:28.582 INFO:teuthology.orchestra.run.smithi071.stdout: openblas x86_64 0.3.15-4.el8 appstream 4.7 M 2024-02-20T12:47:28.582 INFO:teuthology.orchestra.run.smithi071.stdout: openblas-threads x86_64 0.3.15-4.el8 appstream 4.9 M 2024-02-20T12:47:28.582 INFO:teuthology.orchestra.run.smithi071.stdout: python3-numpy x86_64 1:1.14.3-10.el8 appstream 3.7 M 2024-02-20T12:47:28.582 INFO:teuthology.orchestra.run.smithi071.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 appstream 225 k 2024-02-20T12:47:28.582 INFO:teuthology.orchestra.run.smithi071.stdout: python3-scipy x86_64 1.0.0-21.module_el8+762+77bd8591 appstream 14 M 2024-02-20T12:47:28.582 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-20T12:47:28.582 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction Summary 2024-02-20T12:47:28.582 INFO:teuthology.orchestra.run.smithi071.stdout:============================================================================================ 2024-02-20T12:47:28.583 INFO:teuthology.orchestra.run.smithi071.stdout:Install 8 Packages 2024-02-20T12:47:28.583 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-20T12:47:28.584 INFO:teuthology.orchestra.run.smithi071.stdout:Total download size: 37 M 2024-02-20T12:47:28.584 INFO:teuthology.orchestra.run.smithi071.stdout:Installed size: 210 M 2024-02-20T12:47:28.584 INFO:teuthology.orchestra.run.smithi071.stdout:Downloading Packages: 2024-02-20T12:47:28.965 INFO:teuthology.orchestra.run.smithi064.stdout:Dependencies resolved. 2024-02-20T12:47:28.966 INFO:teuthology.orchestra.run.smithi064.stdout:================================================================================ 2024-02-20T12:47:28.966 INFO:teuthology.orchestra.run.smithi064.stdout: Package Arch Version Repository Size 2024-02-20T12:47:28.966 INFO:teuthology.orchestra.run.smithi064.stdout:================================================================================ 2024-02-20T12:47:28.966 INFO:teuthology.orchestra.run.smithi064.stdout:Installing: 2024-02-20T12:47:28.966 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mgr-dashboard noarch 2:16.2.14-548.g81bd20d6.el8 ceph-noarch 3.1 M 2024-02-20T12:47:28.967 INFO:teuthology.orchestra.run.smithi064.stdout:Installing dependencies: 2024-02-20T12:47:28.967 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-grafana-dashboards noarch 2:16.2.14-548.g81bd20d6.el8 ceph-noarch 24 k 2024-02-20T12:47:28.967 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-prometheus-alerts noarch 2:16.2.14-548.g81bd20d6.el8 ceph-noarch 15 k 2024-02-20T12:47:28.967 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jwt noarch 1.6.1-2.el8 baseos 43 k 2024-02-20T12:47:28.967 INFO:teuthology.orchestra.run.smithi064.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2024-02-20T12:47:28.967 INFO:teuthology.orchestra.run.smithi064.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2024-02-20T12:47:28.967 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-20T12:47:28.967 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction Summary 2024-02-20T12:47:28.967 INFO:teuthology.orchestra.run.smithi064.stdout:================================================================================ 2024-02-20T12:47:28.967 INFO:teuthology.orchestra.run.smithi064.stdout:Install 6 Packages 2024-02-20T12:47:28.967 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-20T12:47:28.968 INFO:teuthology.orchestra.run.smithi064.stdout:Total download size: 3.4 M 2024-02-20T12:47:28.968 INFO:teuthology.orchestra.run.smithi064.stdout:Installed size: 22 M 2024-02-20T12:47:28.968 INFO:teuthology.orchestra.run.smithi064.stdout:Downloading Packages: 2024-02-20T12:47:29.453 INFO:teuthology.orchestra.run.smithi064.stdout:(1/6): ceph-grafana-dashboards-16.2.14-548.g81b 157 kB/s | 24 kB 00:00 2024-02-20T12:47:29.562 INFO:teuthology.orchestra.run.smithi064.stdout:(2/6): ceph-prometheus-alerts-16.2.14-548.g81bd 56 kB/s | 15 kB 00:00 2024-02-20T12:47:29.645 INFO:teuthology.orchestra.run.smithi064.stdout:(3/6): python3-jwt-1.6.1-2.el8.noarch.rpm 222 kB/s | 43 kB 00:00 2024-02-20T12:47:29.775 INFO:teuthology.orchestra.run.smithi071.stdout:(1/8): ceph-mgr-diskprediction-local-16.2.14-54 11 MB/s | 8.6 MB 00:00 2024-02-20T12:47:29.854 INFO:teuthology.orchestra.run.smithi064.stdout:(4/6): ceph-mgr-dashboard-16.2.14-548.g81bd20d6 5.6 MB/s | 3.1 MB 00:00 2024-02-20T12:47:29.879 INFO:teuthology.orchestra.run.smithi064.stdout:(5/6): python3-repoze-lru-0.7-6.el8.noarch.rpm 106 kB/s | 34 kB 00:00 2024-02-20T12:47:29.933 INFO:teuthology.orchestra.run.smithi071.stdout:(2/8): openblas-threads-0.3.15-4.el8.x86_64.rpm 5.0 MB/s | 4.9 MB 00:00 2024-02-20T12:47:30.000 INFO:teuthology.orchestra.run.smithi071.stdout:(3/8): openblas-0.3.15-4.el8.x86_64.rpm 4.6 MB/s | 4.7 MB 00:01 2024-02-20T12:47:30.042 INFO:teuthology.orchestra.run.smithi071.stdout:(4/8): python3-numpy-f2py-1.14.3-10.el8.x86_64. 2.0 MB/s | 225 kB 00:00 2024-02-20T12:47:30.063 INFO:teuthology.orchestra.run.smithi064.stdout:(6/6): python3-routes-2.4.1-12.el8.noarch.rpm 471 kB/s | 196 kB 00:00 2024-02-20T12:47:30.065 INFO:teuthology.orchestra.run.smithi064.stdout:-------------------------------------------------------------------------------- 2024-02-20T12:47:30.065 INFO:teuthology.orchestra.run.smithi064.stdout:Total 3.1 MB/s | 3.4 MB 00:01 2024-02-20T12:47:30.077 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction check 2024-02-20T12:47:30.088 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction check succeeded. 2024-02-20T12:47:30.089 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction test 2024-02-20T12:47:30.269 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction test succeeded. 2024-02-20T12:47:30.273 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction 2024-02-20T12:47:30.376 INFO:teuthology.orchestra.run.smithi071.stdout:(5/8): libgfortran-8.5.0-21.el8.x86_64.rpm 1.9 MB/s | 645 kB 00:00 2024-02-20T12:47:30.487 INFO:teuthology.orchestra.run.smithi071.stdout:(6/8): libquadmath-8.5.0-21.el8.x86_64.rpm 1.5 MB/s | 172 kB 00:00 2024-02-20T12:47:30.592 INFO:teuthology.orchestra.run.smithi064.stdout: Preparing : 1/1 2024-02-20T12:47:30.844 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 1/6 2024-02-20T12:47:30.976 INFO:teuthology.orchestra.run.smithi071.stdout:(7/8): python3-scipy-1.0.0-21.module_el8+762+77 15 MB/s | 14 MB 00:00 2024-02-20T12:47:31.163 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 2/6 2024-02-20T12:47:31.388 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 3/6 2024-02-20T12:47:31.543 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-prometheus-alerts-2:16.2.14-548.g81bd20d6.el8.n 4/6 2024-02-20T12:47:32.201 INFO:teuthology.orchestra.run.smithi071.stdout:(8/8): python3-numpy-1.14.3-10.el8.x86_64.rpm 1.5 MB/s | 3.7 MB 00:02 2024-02-20T12:47:32.202 INFO:teuthology.orchestra.run.smithi071.stdout:-------------------------------------------------------------------------------- 2024-02-20T12:47:32.202 INFO:teuthology.orchestra.run.smithi071.stdout:Total 10 MB/s | 37 MB 00:03 2024-02-20T12:47:32.434 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction check 2024-02-20T12:47:32.451 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction check succeeded. 2024-02-20T12:47:32.452 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction test 2024-02-20T12:47:32.661 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-grafana-dashboards-2:16.2.14-548.g81bd20d6.el8. 5/6 2024-02-20T12:47:32.779 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction test succeeded. 2024-02-20T12:47:32.782 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction 2024-02-20T12:47:32.943 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-mgr-dashboard-2:16.2.14-548.g81bd20d6.el8.noarc 6/6 2024-02-20T12:47:33.276 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-mgr-dashboard-2:16.2.14-548.g81bd20d6.el8.noarc 6/6 2024-02-20T12:47:33.277 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-grafana-dashboards-2:16.2.14-548.g81bd20d6.el8. 1/6 2024-02-20T12:47:33.277 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-mgr-dashboard-2:16.2.14-548.g81bd20d6.el8.noarc 2/6 2024-02-20T12:47:33.277 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-prometheus-alerts-2:16.2.14-548.g81bd20d6.el8.n 3/6 2024-02-20T12:47:33.277 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 4/6 2024-02-20T12:47:33.277 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 5/6 2024-02-20T12:47:33.376 INFO:teuthology.orchestra.run.smithi071.stdout: Preparing : 1/1 2024-02-20T12:47:33.520 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : libquadmath-8.5.0-21.el8.x86_64 1/8 2024-02-20T12:47:33.634 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: libquadmath-8.5.0-21.el8.x86_64 1/8 2024-02-20T12:47:33.720 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 6/6 2024-02-20T12:47:33.721 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-20T12:47:33.721 INFO:teuthology.orchestra.run.smithi064.stdout:Installed: 2024-02-20T12:47:33.721 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-grafana-dashboards-2:16.2.14-548.g81bd20d6.el8.noarch 2024-02-20T12:47:33.721 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mgr-dashboard-2:16.2.14-548.g81bd20d6.el8.noarch 2024-02-20T12:47:33.721 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-prometheus-alerts-2:16.2.14-548.g81bd20d6.el8.noarch 2024-02-20T12:47:33.721 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jwt-1.6.1-2.el8.noarch 2024-02-20T12:47:33.721 INFO:teuthology.orchestra.run.smithi064.stdout: python3-repoze-lru-0.7-6.el8.noarch 2024-02-20T12:47:33.721 INFO:teuthology.orchestra.run.smithi064.stdout: python3-routes-2.4.1-12.el8.noarch 2024-02-20T12:47:33.721 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-20T12:47:33.721 INFO:teuthology.orchestra.run.smithi064.stdout:Complete! 2024-02-20T12:47:33.811 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : libgfortran-8.5.0-21.el8.x86_64 2/8 2024-02-20T12:47:33.846 DEBUG:teuthology.orchestra.run.smithi064:> sudo yum -y install ceph-mgr-diskprediction-local 2024-02-20T12:47:34.412 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: libgfortran-8.5.0-21.el8.x86_64 2/8 2024-02-20T12:47:34.633 INFO:teuthology.orchestra.run.smithi064.stdout:Last metadata expiration check: 0:01:40 ago on Tue 20 Feb 2024 12:45:54 PM UTC. 2024-02-20T12:47:35.294 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : openblas-0.3.15-4.el8.x86_64 3/8 2024-02-20T12:47:35.916 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: openblas-0.3.15-4.el8.x86_64 3/8 2024-02-20T12:47:36.069 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : openblas-threads-0.3.15-4.el8.x86_64 4/8 2024-02-20T12:47:36.295 INFO:teuthology.orchestra.run.smithi064.stdout:Dependencies resolved. 2024-02-20T12:47:36.296 INFO:teuthology.orchestra.run.smithi064.stdout:============================================================================================ 2024-02-20T12:47:36.296 INFO:teuthology.orchestra.run.smithi064.stdout: Package Arch Version Repository Size 2024-02-20T12:47:36.296 INFO:teuthology.orchestra.run.smithi064.stdout:============================================================================================ 2024-02-20T12:47:36.296 INFO:teuthology.orchestra.run.smithi064.stdout:Installing: 2024-02-20T12:47:36.296 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mgr-diskprediction-local noarch 2:16.2.14-548.g81bd20d6.el8 ceph-noarch 8.6 M 2024-02-20T12:47:36.296 INFO:teuthology.orchestra.run.smithi064.stdout:Installing dependencies: 2024-02-20T12:47:36.296 INFO:teuthology.orchestra.run.smithi064.stdout: libgfortran x86_64 8.5.0-21.el8 baseos 645 k 2024-02-20T12:47:36.297 INFO:teuthology.orchestra.run.smithi064.stdout: libquadmath x86_64 8.5.0-21.el8 baseos 172 k 2024-02-20T12:47:36.297 INFO:teuthology.orchestra.run.smithi064.stdout: openblas x86_64 0.3.15-4.el8 appstream 4.7 M 2024-02-20T12:47:36.297 INFO:teuthology.orchestra.run.smithi064.stdout: openblas-threads x86_64 0.3.15-4.el8 appstream 4.9 M 2024-02-20T12:47:36.297 INFO:teuthology.orchestra.run.smithi064.stdout: python3-numpy x86_64 1:1.14.3-10.el8 appstream 3.7 M 2024-02-20T12:47:36.297 INFO:teuthology.orchestra.run.smithi064.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 appstream 225 k 2024-02-20T12:47:36.297 INFO:teuthology.orchestra.run.smithi064.stdout: python3-scipy x86_64 1.0.0-21.module_el8+762+77bd8591 appstream 14 M 2024-02-20T12:47:36.297 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-20T12:47:36.297 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction Summary 2024-02-20T12:47:36.297 INFO:teuthology.orchestra.run.smithi064.stdout:============================================================================================ 2024-02-20T12:47:36.297 INFO:teuthology.orchestra.run.smithi064.stdout:Install 8 Packages 2024-02-20T12:47:36.297 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-20T12:47:36.298 INFO:teuthology.orchestra.run.smithi064.stdout:Total download size: 37 M 2024-02-20T12:47:36.298 INFO:teuthology.orchestra.run.smithi064.stdout:Installed size: 210 M 2024-02-20T12:47:36.298 INFO:teuthology.orchestra.run.smithi064.stdout:Downloading Packages: 2024-02-20T12:47:36.754 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: openblas-threads-0.3.15-4.el8.x86_64 4/8 2024-02-20T12:47:37.089 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 5/8 2024-02-20T12:47:37.327 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 6/8 2024-02-20T12:47:37.475 INFO:teuthology.orchestra.run.smithi064.stdout:(1/8): ceph-mgr-diskprediction-local-16.2.14-54 9.1 MB/s | 8.6 MB 00:00 2024-02-20T12:47:37.553 INFO:teuthology.orchestra.run.smithi064.stdout:(2/8): openblas-0.3.15-4.el8.x86_64.rpm 4.6 MB/s | 4.7 MB 00:01 2024-02-20T12:47:37.620 INFO:teuthology.orchestra.run.smithi064.stdout:(3/8): openblas-threads-0.3.15-4.el8.x86_64.rpm 4.4 MB/s | 4.9 MB 00:01 2024-02-20T12:47:37.704 INFO:teuthology.orchestra.run.smithi064.stdout:(4/8): python3-numpy-f2py-1.14.3-10.el8.x86_64. 1.5 MB/s | 225 kB 00:00 2024-02-20T12:47:37.812 INFO:teuthology.orchestra.run.smithi064.stdout:(5/8): libgfortran-8.5.0-21.el8.x86_64.rpm 5.8 MB/s | 645 kB 00:00 2024-02-20T12:47:37.846 INFO:teuthology.orchestra.run.smithi064.stdout:(6/8): libquadmath-8.5.0-21.el8.x86_64.rpm 5.1 MB/s | 172 kB 00:00 2024-02-20T12:47:38.572 INFO:teuthology.orchestra.run.smithi064.stdout:(7/8): python3-numpy-1.14.3-10.el8.x86_64.rpm 3.4 MB/s | 3.7 MB 00:01 2024-02-20T12:47:38.988 INFO:teuthology.orchestra.run.smithi064.stdout:(8/8): python3-scipy-1.0.0-21.module_el8+762+77 10 MB/s | 14 MB 00:01 2024-02-20T12:47:38.989 INFO:teuthology.orchestra.run.smithi064.stdout:-------------------------------------------------------------------------------- 2024-02-20T12:47:38.989 INFO:teuthology.orchestra.run.smithi064.stdout:Total 14 MB/s | 37 MB 00:02 2024-02-20T12:47:39.097 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 6/8 2024-02-20T12:47:39.224 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction check 2024-02-20T12:47:39.243 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction check succeeded. 2024-02-20T12:47:39.243 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction test 2024-02-20T12:47:39.620 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction test succeeded. 2024-02-20T12:47:39.623 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction 2024-02-20T12:47:40.243 INFO:teuthology.orchestra.run.smithi064.stdout: Preparing : 1/1 2024-02-20T12:47:40.367 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : libquadmath-8.5.0-21.el8.x86_64 1/8 2024-02-20T12:47:40.427 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-scipy-1.0.0-21.module_el8+762+77bd8591.x86_6 7/8 2024-02-20T12:47:40.552 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: libquadmath-8.5.0-21.el8.x86_64 1/8 2024-02-20T12:47:40.676 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-mgr-diskprediction-local-2:16.2.14-548.g81bd20d 8/8 2024-02-20T12:47:40.718 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : libgfortran-8.5.0-21.el8.x86_64 2/8 2024-02-20T12:47:40.988 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:16.2.14-548.g81bd20d 8/8 2024-02-20T12:47:40.988 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-mgr-diskprediction-local-2:16.2.14-548.g81bd20d 1/8 2024-02-20T12:47:40.988 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : openblas-0.3.15-4.el8.x86_64 2/8 2024-02-20T12:47:40.988 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : openblas-threads-0.3.15-4.el8.x86_64 3/8 2024-02-20T12:47:40.989 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 4/8 2024-02-20T12:47:40.989 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 5/8 2024-02-20T12:47:40.989 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-scipy-1.0.0-21.module_el8+762+77bd8591.x86_6 6/8 2024-02-20T12:47:40.989 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : libgfortran-8.5.0-21.el8.x86_64 7/8 2024-02-20T12:47:41.354 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: libgfortran-8.5.0-21.el8.x86_64 2/8 2024-02-20T12:47:41.389 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : libquadmath-8.5.0-21.el8.x86_64 8/8 2024-02-20T12:47:41.389 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-20T12:47:41.389 INFO:teuthology.orchestra.run.smithi071.stdout:Installed: 2024-02-20T12:47:41.389 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-mgr-diskprediction-local-2:16.2.14-548.g81bd20d6.el8.noarch 2024-02-20T12:47:41.389 INFO:teuthology.orchestra.run.smithi071.stdout: libgfortran-8.5.0-21.el8.x86_64 2024-02-20T12:47:41.389 INFO:teuthology.orchestra.run.smithi071.stdout: libquadmath-8.5.0-21.el8.x86_64 2024-02-20T12:47:41.389 INFO:teuthology.orchestra.run.smithi071.stdout: openblas-0.3.15-4.el8.x86_64 2024-02-20T12:47:41.390 INFO:teuthology.orchestra.run.smithi071.stdout: openblas-threads-0.3.15-4.el8.x86_64 2024-02-20T12:47:41.390 INFO:teuthology.orchestra.run.smithi071.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2024-02-20T12:47:41.390 INFO:teuthology.orchestra.run.smithi071.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2024-02-20T12:47:41.390 INFO:teuthology.orchestra.run.smithi071.stdout: python3-scipy-1.0.0-21.module_el8+762+77bd8591.x86_64 2024-02-20T12:47:41.390 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-20T12:47:41.390 INFO:teuthology.orchestra.run.smithi071.stdout:Complete! 2024-02-20T12:47:41.510 DEBUG:teuthology.orchestra.run.smithi071:> sudo yum -y install ceph-mgr-rook 2024-02-20T12:47:41.542 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : openblas-0.3.15-4.el8.x86_64 3/8 2024-02-20T12:47:42.175 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: openblas-0.3.15-4.el8.x86_64 3/8 2024-02-20T12:47:42.262 INFO:teuthology.orchestra.run.smithi071.stdout:Last metadata expiration check: 0:01:51 ago on Tue 20 Feb 2024 12:45:51 PM UTC. 2024-02-20T12:47:43.090 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : openblas-threads-0.3.15-4.el8.x86_64 4/8 2024-02-20T12:47:43.783 INFO:teuthology.orchestra.run.smithi071.stdout:Dependencies resolved. 2024-02-20T12:47:43.784 INFO:teuthology.orchestra.run.smithi071.stdout:================================================================================ 2024-02-20T12:47:43.784 INFO:teuthology.orchestra.run.smithi071.stdout: Package Arch Version Repository Size 2024-02-20T12:47:43.785 INFO:teuthology.orchestra.run.smithi071.stdout:================================================================================ 2024-02-20T12:47:43.785 INFO:teuthology.orchestra.run.smithi071.stdout:Installing: 2024-02-20T12:47:43.785 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-mgr-rook noarch 2:16.2.14-548.g81bd20d6.el8 ceph-noarch 27 k 2024-02-20T12:47:43.785 INFO:teuthology.orchestra.run.smithi071.stdout:Installing dependencies: 2024-02-20T12:47:43.785 INFO:teuthology.orchestra.run.smithi071.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2024-02-20T12:47:43.785 INFO:teuthology.orchestra.run.smithi071.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2024-02-20T12:47:43.785 INFO:teuthology.orchestra.run.smithi071.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2024-02-20T12:47:43.785 INFO:teuthology.orchestra.run.smithi071.stdout: python3-jsonpatch noarch 1.21-2.el8 appstream 27 k 2024-02-20T12:47:43.785 INFO:teuthology.orchestra.run.smithi071.stdout: python3-jsonpointer noarch 1.10-11.el8 appstream 20 k 2024-02-20T12:47:43.785 INFO:teuthology.orchestra.run.smithi071.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2024-02-20T12:47:43.785 INFO:teuthology.orchestra.run.smithi071.stdout: python3-oauthlib noarch 2.1.0-1.el8 baseos 155 k 2024-02-20T12:47:43.785 INFO:teuthology.orchestra.run.smithi071.stdout: python3-pyasn1 noarch 0.3.7-6.el8 appstream 126 k 2024-02-20T12:47:43.785 INFO:teuthology.orchestra.run.smithi071.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 appstream 110 k 2024-02-20T12:47:43.786 INFO:teuthology.orchestra.run.smithi071.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 baseos 43 k 2024-02-20T12:47:43.786 INFO:teuthology.orchestra.run.smithi071.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2024-02-20T12:47:43.786 INFO:teuthology.orchestra.run.smithi071.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2024-02-20T12:47:43.786 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-20T12:47:43.786 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction Summary 2024-02-20T12:47:43.786 INFO:teuthology.orchestra.run.smithi071.stdout:================================================================================ 2024-02-20T12:47:43.786 INFO:teuthology.orchestra.run.smithi071.stdout:Install 13 Packages 2024-02-20T12:47:43.786 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-20T12:47:43.787 INFO:teuthology.orchestra.run.smithi071.stdout:Total download size: 1.9 M 2024-02-20T12:47:43.787 INFO:teuthology.orchestra.run.smithi071.stdout:Installed size: 24 M 2024-02-20T12:47:43.787 INFO:teuthology.orchestra.run.smithi071.stdout:Downloading Packages: 2024-02-20T12:47:43.952 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: openblas-threads-0.3.15-4.el8.x86_64 4/8 2024-02-20T12:47:44.342 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 5/8 2024-02-20T12:47:44.632 INFO:teuthology.orchestra.run.smithi071.stdout:(1/13): ceph-mgr-rook-16.2.14-548.g81bd20d6.el8 101 kB/s | 27 kB 00:00 2024-02-20T12:47:44.648 INFO:teuthology.orchestra.run.smithi071.stdout:(2/13): python3-jsonpatch-1.21-2.el8.noarch.rpm 92 kB/s | 27 kB 00:00 2024-02-20T12:47:44.659 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 6/8 2024-02-20T12:47:44.665 INFO:teuthology.orchestra.run.smithi071.stdout:(3/13): python3-jsonpointer-1.10-11.el8.noarch. 65 kB/s | 20 kB 00:00 2024-02-20T12:47:44.857 INFO:teuthology.orchestra.run.smithi071.stdout:(4/13): python3-oauthlib-2.1.0-1.el8.noarch.rpm 808 kB/s | 155 kB 00:00 2024-02-20T12:47:44.883 INFO:teuthology.orchestra.run.smithi071.stdout:(5/13): python3-pyasn1-0.3.7-6.el8.noarch.rpm 504 kB/s | 126 kB 00:00 2024-02-20T12:47:44.908 INFO:teuthology.orchestra.run.smithi071.stdout:(6/13): python3-pyasn1-modules-0.3.7-6.el8.noar 454 kB/s | 110 kB 00:00 2024-02-20T12:47:44.933 INFO:teuthology.orchestra.run.smithi071.stdout:(7/13): python3-requests-oauthlib-1.0.0-1.el8.n 564 kB/s | 43 kB 00:00 2024-02-20T12:47:45.050 INFO:teuthology.orchestra.run.smithi071.stdout:(8/13): python3-certifi-2018.10.15-7.el8.noarch 115 kB/s | 16 kB 00:00 2024-02-20T12:47:45.075 INFO:teuthology.orchestra.run.smithi071.stdout:(9/13): python3-cachetools-3.1.1-4.el8.noarch.r 171 kB/s | 33 kB 00:00 2024-02-20T12:47:45.101 INFO:teuthology.orchestra.run.smithi071.stdout:(10/13): python3-google-auth-1.1.1-10.el8.noarc 486 kB/s | 81 kB 00:00 2024-02-20T12:47:45.184 INFO:teuthology.orchestra.run.smithi071.stdout:(11/13): python3-websocket-client-0.56.0-5.el8. 737 kB/s | 61 kB 00:00 2024-02-20T12:47:45.209 INFO:teuthology.orchestra.run.smithi071.stdout:(12/13): python3-rsa-4.9-2.el8.noarch.rpm 480 kB/s | 64 kB 00:00 2024-02-20T12:47:45.659 INFO:teuthology.orchestra.run.smithi071.stdout:(13/13): python3-kubernetes-11.0.0-6.el8.noarch 1.9 MB/s | 1.2 MB 00:00 2024-02-20T12:47:45.659 INFO:teuthology.orchestra.run.smithi071.stdout:-------------------------------------------------------------------------------- 2024-02-20T12:47:45.660 INFO:teuthology.orchestra.run.smithi071.stdout:Total 1.0 MB/s | 1.9 MB 00:01 2024-02-20T12:47:45.720 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction check 2024-02-20T12:47:45.736 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction check succeeded. 2024-02-20T12:47:45.736 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction test 2024-02-20T12:47:45.823 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction test succeeded. 2024-02-20T12:47:45.827 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction 2024-02-20T12:47:46.476 INFO:teuthology.orchestra.run.smithi071.stdout: Preparing : 1/1 2024-02-20T12:47:46.554 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 6/8 2024-02-20T12:47:46.727 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 1/13 2024-02-20T12:47:46.996 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 2/13 2024-02-20T12:47:47.227 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-rsa-4.9-2.el8.noarch 3/13 2024-02-20T12:47:47.490 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 4/13 2024-02-20T12:47:47.764 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 5/13 2024-02-20T12:47:47.953 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-scipy-1.0.0-21.module_el8+762+77bd8591.x86_6 7/8 2024-02-20T12:47:48.005 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 6/13 2024-02-20T12:47:48.176 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-mgr-diskprediction-local-2:16.2.14-548.g81bd20d 8/8 2024-02-20T12:47:48.333 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 7/13 2024-02-20T12:47:48.542 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:16.2.14-548.g81bd20d 8/8 2024-02-20T12:47:48.543 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-mgr-diskprediction-local-2:16.2.14-548.g81bd20d 1/8 2024-02-20T12:47:48.543 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : openblas-0.3.15-4.el8.x86_64 2/8 2024-02-20T12:47:48.543 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : openblas-threads-0.3.15-4.el8.x86_64 3/8 2024-02-20T12:47:48.543 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 4/8 2024-02-20T12:47:48.543 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 5/8 2024-02-20T12:47:48.543 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-scipy-1.0.0-21.module_el8+762+77bd8591.x86_6 6/8 2024-02-20T12:47:48.543 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libgfortran-8.5.0-21.el8.x86_64 7/8 2024-02-20T12:47:48.546 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 8/13 2024-02-20T12:47:49.008 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libquadmath-8.5.0-21.el8.x86_64 8/8 2024-02-20T12:47:49.009 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-20T12:47:49.009 INFO:teuthology.orchestra.run.smithi064.stdout:Installed: 2024-02-20T12:47:49.009 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mgr-diskprediction-local-2:16.2.14-548.g81bd20d6.el8.noarch 2024-02-20T12:47:49.009 INFO:teuthology.orchestra.run.smithi064.stdout: libgfortran-8.5.0-21.el8.x86_64 2024-02-20T12:47:49.009 INFO:teuthology.orchestra.run.smithi064.stdout: libquadmath-8.5.0-21.el8.x86_64 2024-02-20T12:47:49.009 INFO:teuthology.orchestra.run.smithi064.stdout: openblas-0.3.15-4.el8.x86_64 2024-02-20T12:47:49.009 INFO:teuthology.orchestra.run.smithi064.stdout: openblas-threads-0.3.15-4.el8.x86_64 2024-02-20T12:47:49.009 INFO:teuthology.orchestra.run.smithi064.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2024-02-20T12:47:49.009 INFO:teuthology.orchestra.run.smithi064.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2024-02-20T12:47:49.009 INFO:teuthology.orchestra.run.smithi064.stdout: python3-scipy-1.0.0-21.module_el8+762+77bd8591.x86_64 2024-02-20T12:47:49.009 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-20T12:47:49.009 INFO:teuthology.orchestra.run.smithi064.stdout:Complete! 2024-02-20T12:47:49.153 DEBUG:teuthology.orchestra.run.smithi064:> sudo yum -y install ceph-mgr-rook 2024-02-20T12:47:49.571 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 9/13 2024-02-20T12:47:49.881 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 10/13 2024-02-20T12:47:49.945 INFO:teuthology.orchestra.run.smithi064.stdout:Last metadata expiration check: 0:01:55 ago on Tue 20 Feb 2024 12:45:54 PM UTC. 2024-02-20T12:47:50.115 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 11/13 2024-02-20T12:47:50.373 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 12/13 2024-02-20T12:47:50.532 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-mgr-rook-2:16.2.14-548.g81bd20d6.el8.noarch 13/13 2024-02-20T12:47:50.856 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: ceph-mgr-rook-2:16.2.14-548.g81bd20d6.el8.noarch 13/13 2024-02-20T12:47:50.856 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-mgr-rook-2:16.2.14-548.g81bd20d6.el8.noarch 1/13 2024-02-20T12:47:50.856 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 2/13 2024-02-20T12:47:50.857 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 3/13 2024-02-20T12:47:50.857 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 4/13 2024-02-20T12:47:50.857 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 5/13 2024-02-20T12:47:50.857 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 6/13 2024-02-20T12:47:50.857 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 7/13 2024-02-20T12:47:50.857 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 8/13 2024-02-20T12:47:50.858 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 9/13 2024-02-20T12:47:50.858 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 10/13 2024-02-20T12:47:50.858 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 11/13 2024-02-20T12:47:50.858 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 12/13 2024-02-20T12:47:51.338 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 13/13 2024-02-20T12:47:51.338 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-20T12:47:51.338 INFO:teuthology.orchestra.run.smithi071.stdout:Installed: 2024-02-20T12:47:51.338 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-mgr-rook-2:16.2.14-548.g81bd20d6.el8.noarch 2024-02-20T12:47:51.338 INFO:teuthology.orchestra.run.smithi071.stdout: python3-cachetools-3.1.1-4.el8.noarch 2024-02-20T12:47:51.338 INFO:teuthology.orchestra.run.smithi071.stdout: python3-certifi-2018.10.15-7.el8.noarch 2024-02-20T12:47:51.339 INFO:teuthology.orchestra.run.smithi071.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2024-02-20T12:47:51.339 INFO:teuthology.orchestra.run.smithi071.stdout: python3-jsonpatch-1.21-2.el8.noarch 2024-02-20T12:47:51.339 INFO:teuthology.orchestra.run.smithi071.stdout: python3-jsonpointer-1.10-11.el8.noarch 2024-02-20T12:47:51.339 INFO:teuthology.orchestra.run.smithi071.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2024-02-20T12:47:51.339 INFO:teuthology.orchestra.run.smithi071.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2024-02-20T12:47:51.339 INFO:teuthology.orchestra.run.smithi071.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2024-02-20T12:47:51.339 INFO:teuthology.orchestra.run.smithi071.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2024-02-20T12:47:51.339 INFO:teuthology.orchestra.run.smithi071.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2024-02-20T12:47:51.339 INFO:teuthology.orchestra.run.smithi071.stdout: python3-rsa-4.9-2.el8.noarch 2024-02-20T12:47:51.339 INFO:teuthology.orchestra.run.smithi071.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2024-02-20T12:47:51.339 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-20T12:47:51.339 INFO:teuthology.orchestra.run.smithi071.stdout:Complete! 2024-02-20T12:47:51.470 DEBUG:teuthology.orchestra.run.smithi071:> sudo yum -y install ceph-mgr-cephadm 2024-02-20T12:47:51.610 INFO:teuthology.orchestra.run.smithi064.stdout:Dependencies resolved. 2024-02-20T12:47:51.612 INFO:teuthology.orchestra.run.smithi064.stdout:================================================================================ 2024-02-20T12:47:51.612 INFO:teuthology.orchestra.run.smithi064.stdout: Package Arch Version Repository Size 2024-02-20T12:47:51.612 INFO:teuthology.orchestra.run.smithi064.stdout:================================================================================ 2024-02-20T12:47:51.612 INFO:teuthology.orchestra.run.smithi064.stdout:Installing: 2024-02-20T12:47:51.612 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mgr-rook noarch 2:16.2.14-548.g81bd20d6.el8 ceph-noarch 27 k 2024-02-20T12:47:51.612 INFO:teuthology.orchestra.run.smithi064.stdout:Installing dependencies: 2024-02-20T12:47:51.612 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2024-02-20T12:47:51.613 INFO:teuthology.orchestra.run.smithi064.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2024-02-20T12:47:51.613 INFO:teuthology.orchestra.run.smithi064.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2024-02-20T12:47:51.613 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jsonpatch noarch 1.21-2.el8 appstream 27 k 2024-02-20T12:47:51.613 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jsonpointer noarch 1.10-11.el8 appstream 20 k 2024-02-20T12:47:51.613 INFO:teuthology.orchestra.run.smithi064.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2024-02-20T12:47:51.613 INFO:teuthology.orchestra.run.smithi064.stdout: python3-oauthlib noarch 2.1.0-1.el8 baseos 155 k 2024-02-20T12:47:51.613 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pyasn1 noarch 0.3.7-6.el8 appstream 126 k 2024-02-20T12:47:51.613 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 appstream 110 k 2024-02-20T12:47:51.613 INFO:teuthology.orchestra.run.smithi064.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 baseos 43 k 2024-02-20T12:47:51.613 INFO:teuthology.orchestra.run.smithi064.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2024-02-20T12:47:51.613 INFO:teuthology.orchestra.run.smithi064.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2024-02-20T12:47:51.614 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-20T12:47:51.614 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction Summary 2024-02-20T12:47:51.614 INFO:teuthology.orchestra.run.smithi064.stdout:================================================================================ 2024-02-20T12:47:51.614 INFO:teuthology.orchestra.run.smithi064.stdout:Install 13 Packages 2024-02-20T12:47:51.614 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-20T12:47:51.614 INFO:teuthology.orchestra.run.smithi064.stdout:Total download size: 1.9 M 2024-02-20T12:47:51.615 INFO:teuthology.orchestra.run.smithi064.stdout:Installed size: 24 M 2024-02-20T12:47:51.615 INFO:teuthology.orchestra.run.smithi064.stdout:Downloading Packages: 2024-02-20T12:47:52.195 INFO:teuthology.orchestra.run.smithi064.stdout:(1/13): ceph-mgr-rook-16.2.14-548.g81bd20d6.el8 176 kB/s | 27 kB 00:00 2024-02-20T12:47:52.224 INFO:teuthology.orchestra.run.smithi071.stdout:Last metadata expiration check: 0:02:01 ago on Tue 20 Feb 2024 12:45:51 PM UTC. 2024-02-20T12:47:52.286 INFO:teuthology.orchestra.run.smithi064.stdout:(2/13): python3-jsonpointer-1.10-11.el8.noarch. 81 kB/s | 20 kB 00:00 2024-02-20T12:47:52.312 INFO:teuthology.orchestra.run.smithi064.stdout:(3/13): python3-jsonpatch-1.21-2.el8.noarch.rpm 97 kB/s | 27 kB 00:00 2024-02-20T12:47:52.395 INFO:teuthology.orchestra.run.smithi064.stdout:(4/13): python3-oauthlib-2.1.0-1.el8.noarch.rpm 1.8 MB/s | 155 kB 00:00 2024-02-20T12:47:52.459 INFO:teuthology.orchestra.run.smithi064.stdout:(5/13): python3-requests-oauthlib-1.0.0-1.el8.n 675 kB/s | 43 kB 00:00 2024-02-20T12:47:52.479 INFO:teuthology.orchestra.run.smithi064.stdout:(6/13): python3-pyasn1-modules-0.3.7-6.el8.noar 571 kB/s | 110 kB 00:00 2024-02-20T12:47:52.613 INFO:teuthology.orchestra.run.smithi064.stdout:(7/13): python3-pyasn1-0.3.7-6.el8.noarch.rpm 302 kB/s | 126 kB 00:00 2024-02-20T12:47:52.747 INFO:teuthology.orchestra.run.smithi064.stdout:(8/13): python3-certifi-2018.10.15-7.el8.noarch 61 kB/s | 16 kB 00:00 2024-02-20T12:47:52.772 INFO:teuthology.orchestra.run.smithi064.stdout:(9/13): python3-cachetools-3.1.1-4.el8.noarch.r 105 kB/s | 33 kB 00:00 2024-02-20T12:47:52.956 INFO:teuthology.orchestra.run.smithi064.stdout:(10/13): python3-google-auth-1.1.1-10.el8.noarc 237 kB/s | 81 kB 00:00 2024-02-20T12:47:52.972 INFO:teuthology.orchestra.run.smithi064.stdout:(11/13): python3-rsa-4.9-2.el8.noarch.rpm 320 kB/s | 64 kB 00:00 2024-02-20T12:47:53.096 INFO:teuthology.orchestra.run.smithi064.stdout:(12/13): python3-websocket-client-0.56.0-5.el8. 441 kB/s | 61 kB 00:00 2024-02-20T12:47:53.323 INFO:teuthology.orchestra.run.smithi064.stdout:(13/13): python3-kubernetes-11.0.0-6.el8.noarch 2.0 MB/s | 1.2 MB 00:00 2024-02-20T12:47:53.324 INFO:teuthology.orchestra.run.smithi064.stdout:-------------------------------------------------------------------------------- 2024-02-20T12:47:53.325 INFO:teuthology.orchestra.run.smithi064.stdout:Total 1.1 MB/s | 1.9 MB 00:01 2024-02-20T12:47:53.385 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction check 2024-02-20T12:47:53.403 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction check succeeded. 2024-02-20T12:47:53.403 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction test 2024-02-20T12:47:53.499 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction test succeeded. 2024-02-20T12:47:53.502 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction 2024-02-20T12:47:53.766 INFO:teuthology.orchestra.run.smithi071.stdout:Dependencies resolved. 2024-02-20T12:47:53.767 INFO:teuthology.orchestra.run.smithi071.stdout:================================================================================ 2024-02-20T12:47:53.768 INFO:teuthology.orchestra.run.smithi071.stdout: Package Arch Version Repository Size 2024-02-20T12:47:53.768 INFO:teuthology.orchestra.run.smithi071.stdout:================================================================================ 2024-02-20T12:47:53.768 INFO:teuthology.orchestra.run.smithi071.stdout:Installing: 2024-02-20T12:47:53.768 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-mgr-cephadm noarch 2:16.2.14-548.g81bd20d6.el8 ceph-noarch 116 k 2024-02-20T12:47:53.768 INFO:teuthology.orchestra.run.smithi071.stdout:Installing dependencies: 2024-02-20T12:47:53.768 INFO:teuthology.orchestra.run.smithi071.stdout: python3-apipkg noarch 1.5-6.el8 epel 21 k 2024-02-20T12:47:53.768 INFO:teuthology.orchestra.run.smithi071.stdout: python3-babel noarch 2.5.1-7.el8 appstream 4.8 M 2024-02-20T12:47:53.768 INFO:teuthology.orchestra.run.smithi071.stdout: python3-execnet noarch 1.7.1-1.el8 epel 355 k 2024-02-20T12:47:53.768 INFO:teuthology.orchestra.run.smithi071.stdout: python3-jinja2 noarch 2.10.1-4.el8 appstream 578 k 2024-02-20T12:47:53.768 INFO:teuthology.orchestra.run.smithi071.stdout: python3-remoto noarch 1.2.1-1.el8 epel 49 k 2024-02-20T12:47:53.768 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-20T12:47:53.768 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction Summary 2024-02-20T12:47:53.769 INFO:teuthology.orchestra.run.smithi071.stdout:================================================================================ 2024-02-20T12:47:53.769 INFO:teuthology.orchestra.run.smithi071.stdout:Install 6 Packages 2024-02-20T12:47:53.769 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-20T12:47:53.769 INFO:teuthology.orchestra.run.smithi071.stdout:Total download size: 5.9 M 2024-02-20T12:47:53.769 INFO:teuthology.orchestra.run.smithi071.stdout:Installed size: 24 M 2024-02-20T12:47:53.769 INFO:teuthology.orchestra.run.smithi071.stdout:Downloading Packages: 2024-02-20T12:47:54.217 INFO:teuthology.orchestra.run.smithi064.stdout: Preparing : 1/1 2024-02-20T12:47:54.316 INFO:teuthology.orchestra.run.smithi071.stdout:(1/6): ceph-mgr-cephadm-16.2.14-548.g81bd20d6.e 611 kB/s | 116 kB 00:00 2024-02-20T12:47:54.463 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 1/13 2024-02-20T12:47:54.650 INFO:teuthology.orchestra.run.smithi071.stdout:(2/6): python3-apipkg-1.5-6.el8.noarch.rpm 64 kB/s | 21 kB 00:00 2024-02-20T12:47:54.720 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 2/13 2024-02-20T12:47:54.775 INFO:teuthology.orchestra.run.smithi071.stdout:(3/6): python3-jinja2-2.10.1-4.el8.noarch.rpm 891 kB/s | 578 kB 00:00 2024-02-20T12:47:54.909 INFO:teuthology.orchestra.run.smithi071.stdout:(4/6): python3-execnet-1.7.1-1.el8.noarch.rpm 1.3 MB/s | 355 kB 00:00 2024-02-20T12:47:54.995 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-rsa-4.9-2.el8.noarch 3/13 2024-02-20T12:47:55.092 INFO:teuthology.orchestra.run.smithi071.stdout:(5/6): python3-babel-2.5.1-7.el8.noarch.rpm 5.0 MB/s | 4.8 MB 00:00 2024-02-20T12:47:55.117 INFO:teuthology.orchestra.run.smithi071.stdout:(6/6): python3-remoto-1.2.1-1.el8.noarch.rpm 143 kB/s | 49 kB 00:00 2024-02-20T12:47:55.120 INFO:teuthology.orchestra.run.smithi071.stdout:-------------------------------------------------------------------------------- 2024-02-20T12:47:55.120 INFO:teuthology.orchestra.run.smithi071.stdout:Total 4.4 MB/s | 5.9 MB 00:01 2024-02-20T12:47:55.172 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction check 2024-02-20T12:47:55.182 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction check succeeded. 2024-02-20T12:47:55.182 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction test 2024-02-20T12:47:55.282 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 4/13 2024-02-20T12:47:55.283 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction test succeeded. 2024-02-20T12:47:55.286 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction 2024-02-20T12:47:55.549 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 5/13 2024-02-20T12:47:55.580 INFO:teuthology.orchestra.run.smithi071.stdout: Preparing : 1/1 2024-02-20T12:47:55.827 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 6/13 2024-02-20T12:47:55.888 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-apipkg-1.5-6.el8.noarch 1/6 2024-02-20T12:47:56.183 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-execnet-1.7.1-1.el8.noarch 2/6 2024-02-20T12:47:56.197 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 7/13 2024-02-20T12:47:56.465 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 8/13 2024-02-20T12:47:57.452 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-remoto-1.2.1-1.el8.noarch 3/6 2024-02-20T12:47:57.591 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 9/13 2024-02-20T12:47:57.793 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 4/6 2024-02-20T12:47:57.897 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 10/13 2024-02-20T12:47:58.058 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-jinja2-2.10.1-4.el8.noarch 5/6 2024-02-20T12:47:58.141 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 11/13 2024-02-20T12:47:58.306 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-mgr-cephadm-2:16.2.14-548.g81bd20d6.el8.noarch 6/6 2024-02-20T12:47:58.357 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 12/13 2024-02-20T12:47:58.573 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-mgr-rook-2:16.2.14-548.g81bd20d6.el8.noarch 13/13 2024-02-20T12:47:58.596 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: ceph-mgr-cephadm-2:16.2.14-548.g81bd20d6.el8.noarch 6/6 2024-02-20T12:47:58.596 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-mgr-cephadm-2:16.2.14-548.g81bd20d6.el8.noarch 1/6 2024-02-20T12:47:58.596 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 2/6 2024-02-20T12:47:58.596 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-jinja2-2.10.1-4.el8.noarch 3/6 2024-02-20T12:47:58.596 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-apipkg-1.5-6.el8.noarch 4/6 2024-02-20T12:47:58.597 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-execnet-1.7.1-1.el8.noarch 5/6 2024-02-20T12:47:58.889 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-mgr-rook-2:16.2.14-548.g81bd20d6.el8.noarch 13/13 2024-02-20T12:47:58.889 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-mgr-rook-2:16.2.14-548.g81bd20d6.el8.noarch 1/13 2024-02-20T12:47:58.889 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 2/13 2024-02-20T12:47:58.889 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 3/13 2024-02-20T12:47:58.890 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 4/13 2024-02-20T12:47:58.890 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 5/13 2024-02-20T12:47:58.890 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 6/13 2024-02-20T12:47:58.890 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 7/13 2024-02-20T12:47:58.890 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 8/13 2024-02-20T12:47:58.890 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 9/13 2024-02-20T12:47:58.890 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 10/13 2024-02-20T12:47:58.890 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 11/13 2024-02-20T12:47:58.890 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 12/13 2024-02-20T12:47:59.011 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-remoto-1.2.1-1.el8.noarch 6/6 2024-02-20T12:47:59.011 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-20T12:47:59.011 INFO:teuthology.orchestra.run.smithi071.stdout:Installed: 2024-02-20T12:47:59.012 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-mgr-cephadm-2:16.2.14-548.g81bd20d6.el8.noarch 2024-02-20T12:47:59.012 INFO:teuthology.orchestra.run.smithi071.stdout: python3-apipkg-1.5-6.el8.noarch 2024-02-20T12:47:59.012 INFO:teuthology.orchestra.run.smithi071.stdout: python3-babel-2.5.1-7.el8.noarch 2024-02-20T12:47:59.012 INFO:teuthology.orchestra.run.smithi071.stdout: python3-execnet-1.7.1-1.el8.noarch 2024-02-20T12:47:59.012 INFO:teuthology.orchestra.run.smithi071.stdout: python3-jinja2-2.10.1-4.el8.noarch 2024-02-20T12:47:59.012 INFO:teuthology.orchestra.run.smithi071.stdout: python3-remoto-1.2.1-1.el8.noarch 2024-02-20T12:47:59.012 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-20T12:47:59.012 INFO:teuthology.orchestra.run.smithi071.stdout:Complete! 2024-02-20T12:47:59.165 DEBUG:teuthology.orchestra.run.smithi071:> sudo yum -y install ceph-fuse 2024-02-20T12:47:59.391 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 13/13 2024-02-20T12:47:59.391 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-20T12:47:59.391 INFO:teuthology.orchestra.run.smithi064.stdout:Installed: 2024-02-20T12:47:59.391 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mgr-rook-2:16.2.14-548.g81bd20d6.el8.noarch 2024-02-20T12:47:59.391 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cachetools-3.1.1-4.el8.noarch 2024-02-20T12:47:59.391 INFO:teuthology.orchestra.run.smithi064.stdout: python3-certifi-2018.10.15-7.el8.noarch 2024-02-20T12:47:59.391 INFO:teuthology.orchestra.run.smithi064.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2024-02-20T12:47:59.391 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jsonpatch-1.21-2.el8.noarch 2024-02-20T12:47:59.392 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jsonpointer-1.10-11.el8.noarch 2024-02-20T12:47:59.392 INFO:teuthology.orchestra.run.smithi064.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2024-02-20T12:47:59.392 INFO:teuthology.orchestra.run.smithi064.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2024-02-20T12:47:59.392 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2024-02-20T12:47:59.392 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2024-02-20T12:47:59.392 INFO:teuthology.orchestra.run.smithi064.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2024-02-20T12:47:59.392 INFO:teuthology.orchestra.run.smithi064.stdout: python3-rsa-4.9-2.el8.noarch 2024-02-20T12:47:59.392 INFO:teuthology.orchestra.run.smithi064.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2024-02-20T12:47:59.392 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-20T12:47:59.392 INFO:teuthology.orchestra.run.smithi064.stdout:Complete! 2024-02-20T12:47:59.507 DEBUG:teuthology.orchestra.run.smithi064:> sudo yum -y install ceph-mgr-cephadm 2024-02-20T12:47:59.916 INFO:teuthology.orchestra.run.smithi071.stdout:Last metadata expiration check: 0:02:08 ago on Tue 20 Feb 2024 12:45:51 PM UTC. 2024-02-20T12:48:00.302 INFO:teuthology.orchestra.run.smithi064.stdout:Last metadata expiration check: 0:02:06 ago on Tue 20 Feb 2024 12:45:54 PM UTC. 2024-02-20T12:48:01.470 INFO:teuthology.orchestra.run.smithi071.stdout:Dependencies resolved. 2024-02-20T12:48:01.471 INFO:teuthology.orchestra.run.smithi071.stdout:================================================================================ 2024-02-20T12:48:01.471 INFO:teuthology.orchestra.run.smithi071.stdout: Package Arch Version Repository Size 2024-02-20T12:48:01.471 INFO:teuthology.orchestra.run.smithi071.stdout:================================================================================ 2024-02-20T12:48:01.471 INFO:teuthology.orchestra.run.smithi071.stdout:Installing: 2024-02-20T12:48:01.471 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-fuse x86_64 2:16.2.14-548.g81bd20d6.el8 ceph 911 k 2024-02-20T12:48:01.471 INFO:teuthology.orchestra.run.smithi071.stdout:Installing dependencies: 2024-02-20T12:48:01.472 INFO:teuthology.orchestra.run.smithi071.stdout: fuse x86_64 2.9.7-17.el8 baseos 83 k 2024-02-20T12:48:01.472 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-20T12:48:01.472 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction Summary 2024-02-20T12:48:01.472 INFO:teuthology.orchestra.run.smithi071.stdout:================================================================================ 2024-02-20T12:48:01.472 INFO:teuthology.orchestra.run.smithi071.stdout:Install 2 Packages 2024-02-20T12:48:01.472 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-20T12:48:01.472 INFO:teuthology.orchestra.run.smithi071.stdout:Total download size: 994 k 2024-02-20T12:48:01.473 INFO:teuthology.orchestra.run.smithi071.stdout:Installed size: 2.8 M 2024-02-20T12:48:01.473 INFO:teuthology.orchestra.run.smithi071.stdout:Downloading Packages: 2024-02-20T12:48:01.595 INFO:teuthology.orchestra.run.smithi071.stdout:(1/2): fuse-2.9.7-17.el8.x86_64.rpm 1.0 MB/s | 83 kB 00:00 2024-02-20T12:48:01.958 INFO:teuthology.orchestra.run.smithi064.stdout:Dependencies resolved. 2024-02-20T12:48:01.959 INFO:teuthology.orchestra.run.smithi064.stdout:================================================================================ 2024-02-20T12:48:01.959 INFO:teuthology.orchestra.run.smithi064.stdout: Package Arch Version Repository Size 2024-02-20T12:48:01.959 INFO:teuthology.orchestra.run.smithi064.stdout:================================================================================ 2024-02-20T12:48:01.959 INFO:teuthology.orchestra.run.smithi064.stdout:Installing: 2024-02-20T12:48:01.959 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mgr-cephadm noarch 2:16.2.14-548.g81bd20d6.el8 ceph-noarch 116 k 2024-02-20T12:48:01.959 INFO:teuthology.orchestra.run.smithi064.stdout:Installing dependencies: 2024-02-20T12:48:01.960 INFO:teuthology.orchestra.run.smithi064.stdout: python3-apipkg noarch 1.5-6.el8 epel 21 k 2024-02-20T12:48:01.960 INFO:teuthology.orchestra.run.smithi064.stdout: python3-babel noarch 2.5.1-7.el8 appstream 4.8 M 2024-02-20T12:48:01.960 INFO:teuthology.orchestra.run.smithi064.stdout: python3-execnet noarch 1.7.1-1.el8 epel 355 k 2024-02-20T12:48:01.960 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jinja2 noarch 2.10.1-4.el8 appstream 578 k 2024-02-20T12:48:01.960 INFO:teuthology.orchestra.run.smithi064.stdout: python3-remoto noarch 1.2.1-1.el8 epel 49 k 2024-02-20T12:48:01.960 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-20T12:48:01.960 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction Summary 2024-02-20T12:48:01.960 INFO:teuthology.orchestra.run.smithi064.stdout:================================================================================ 2024-02-20T12:48:01.960 INFO:teuthology.orchestra.run.smithi064.stdout:Install 6 Packages 2024-02-20T12:48:01.960 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-20T12:48:01.961 INFO:teuthology.orchestra.run.smithi064.stdout:Total download size: 5.9 M 2024-02-20T12:48:01.961 INFO:teuthology.orchestra.run.smithi064.stdout:Installed size: 24 M 2024-02-20T12:48:01.961 INFO:teuthology.orchestra.run.smithi064.stdout:Downloading Packages: 2024-02-20T12:48:02.112 INFO:teuthology.orchestra.run.smithi071.stdout:(2/2): ceph-fuse-16.2.14-548.g81bd20d6.el8.x86_ 1.5 MB/s | 911 kB 00:00 2024-02-20T12:48:02.112 INFO:teuthology.orchestra.run.smithi071.stdout:-------------------------------------------------------------------------------- 2024-02-20T12:48:02.113 INFO:teuthology.orchestra.run.smithi071.stdout:Total 1.5 MB/s | 994 kB 00:00 2024-02-20T12:48:02.123 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction check 2024-02-20T12:48:02.144 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction check succeeded. 2024-02-20T12:48:02.144 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction test 2024-02-20T12:48:02.234 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction test succeeded. 2024-02-20T12:48:02.237 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction 2024-02-20T12:48:02.427 INFO:teuthology.orchestra.run.smithi071.stdout: Preparing : 1/1 2024-02-20T12:48:02.694 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : fuse-2.9.7-17.el8.x86_64 1/2 2024-02-20T12:48:02.747 INFO:teuthology.orchestra.run.smithi064.stdout:(1/6): ceph-mgr-cephadm-16.2.14-548.g81bd20d6.e 367 kB/s | 116 kB 00:00 2024-02-20T12:48:02.889 INFO:teuthology.orchestra.run.smithi064.stdout:(2/6): python3-apipkg-1.5-6.el8.noarch.rpm 150 kB/s | 21 kB 00:00 2024-02-20T12:48:02.905 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-fuse-2:16.2.14-548.g81bd20d6.el8.x86_64 2/2 2024-02-20T12:48:03.155 INFO:teuthology.orchestra.run.smithi064.stdout:(3/6): python3-execnet-1.7.1-1.el8.noarch.rpm 1.3 MB/s | 355 kB 00:00 2024-02-20T12:48:03.190 INFO:teuthology.orchestra.run.smithi064.stdout:(4/6): python3-jinja2-2.10.1-4.el8.noarch.rpm 762 kB/s | 578 kB 00:00 2024-02-20T12:48:03.232 INFO:teuthology.orchestra.run.smithi064.stdout:(5/6): python3-remoto-1.2.1-1.el8.noarch.rpm 639 kB/s | 49 kB 00:00 2024-02-20T12:48:03.282 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: ceph-fuse-2:16.2.14-548.g81bd20d6.el8.x86_64 2/2 2024-02-20T12:48:03.282 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-fuse-2:16.2.14-548.g81bd20d6.el8.x86_64 1/2 2024-02-20T12:48:03.440 INFO:teuthology.orchestra.run.smithi064.stdout:(6/6): python3-babel-2.5.1-7.el8.noarch.rpm 4.7 MB/s | 4.8 MB 00:01 2024-02-20T12:48:03.441 INFO:teuthology.orchestra.run.smithi064.stdout:-------------------------------------------------------------------------------- 2024-02-20T12:48:03.441 INFO:teuthology.orchestra.run.smithi064.stdout:Total 4.0 MB/s | 5.9 MB 00:01 2024-02-20T12:48:03.496 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction check 2024-02-20T12:48:03.507 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction check succeeded. 2024-02-20T12:48:03.507 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction test 2024-02-20T12:48:03.629 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction test succeeded. 2024-02-20T12:48:03.632 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction 2024-02-20T12:48:03.670 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : fuse-2.9.7-17.el8.x86_64 2/2 2024-02-20T12:48:03.670 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-20T12:48:03.670 INFO:teuthology.orchestra.run.smithi071.stdout:Installed: 2024-02-20T12:48:03.670 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-fuse-2:16.2.14-548.g81bd20d6.el8.x86_64 fuse-2.9.7-17.el8.x86_64 2024-02-20T12:48:03.670 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-20T12:48:03.670 INFO:teuthology.orchestra.run.smithi071.stdout:Complete! 2024-02-20T12:48:03.894 DEBUG:teuthology.orchestra.run.smithi071:> sudo yum -y install librados-devel 2024-02-20T12:48:04.286 INFO:teuthology.orchestra.run.smithi064.stdout: Preparing : 1/1 2024-02-20T12:48:04.662 INFO:teuthology.orchestra.run.smithi071.stdout:Last metadata expiration check: 0:02:13 ago on Tue 20 Feb 2024 12:45:51 PM UTC. 2024-02-20T12:48:04.880 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-apipkg-1.5-6.el8.noarch 1/6 2024-02-20T12:48:05.187 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-execnet-1.7.1-1.el8.noarch 2/6 2024-02-20T12:48:06.190 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-remoto-1.2.1-1.el8.noarch 3/6 2024-02-20T12:48:06.216 INFO:teuthology.orchestra.run.smithi071.stdout:Dependencies resolved. 2024-02-20T12:48:06.217 INFO:teuthology.orchestra.run.smithi071.stdout:================================================================================ 2024-02-20T12:48:06.217 INFO:teuthology.orchestra.run.smithi071.stdout: Package Arch Version Repo Size 2024-02-20T12:48:06.217 INFO:teuthology.orchestra.run.smithi071.stdout:================================================================================ 2024-02-20T12:48:06.217 INFO:teuthology.orchestra.run.smithi071.stdout:Installing: 2024-02-20T12:48:06.217 INFO:teuthology.orchestra.run.smithi071.stdout: librados-devel x86_64 2:16.2.14-548.g81bd20d6.el8 ceph 131 k 2024-02-20T12:48:06.217 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-20T12:48:06.217 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction Summary 2024-02-20T12:48:06.217 INFO:teuthology.orchestra.run.smithi071.stdout:================================================================================ 2024-02-20T12:48:06.217 INFO:teuthology.orchestra.run.smithi071.stdout:Install 1 Package 2024-02-20T12:48:06.217 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-20T12:48:06.218 INFO:teuthology.orchestra.run.smithi071.stdout:Total download size: 131 k 2024-02-20T12:48:06.218 INFO:teuthology.orchestra.run.smithi071.stdout:Installed size: 432 k 2024-02-20T12:48:06.218 INFO:teuthology.orchestra.run.smithi071.stdout:Downloading Packages: 2024-02-20T12:48:06.515 INFO:teuthology.orchestra.run.smithi071.stdout:librados-devel-16.2.14-548.g81bd20d6.el8.x86_64 442 kB/s | 131 kB 00:00 2024-02-20T12:48:06.515 INFO:teuthology.orchestra.run.smithi071.stdout:-------------------------------------------------------------------------------- 2024-02-20T12:48:06.515 INFO:teuthology.orchestra.run.smithi071.stdout:Total 440 kB/s | 131 kB 00:00 2024-02-20T12:48:06.516 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction check 2024-02-20T12:48:06.521 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction check succeeded. 2024-02-20T12:48:06.521 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction test 2024-02-20T12:48:06.571 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction test succeeded. 2024-02-20T12:48:06.574 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction 2024-02-20T12:48:06.577 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 4/6 2024-02-20T12:48:06.694 INFO:teuthology.orchestra.run.smithi071.stdout: Preparing : 1/1 2024-02-20T12:48:06.866 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-jinja2-2.10.1-4.el8.noarch 5/6 2024-02-20T12:48:06.953 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : librados-devel-2:16.2.14-548.g81bd20d6.el8.x86_64 1/1 2024-02-20T12:48:07.122 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-mgr-cephadm-2:16.2.14-548.g81bd20d6.el8.noarch 6/6 2024-02-20T12:48:07.435 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-mgr-cephadm-2:16.2.14-548.g81bd20d6.el8.noarch 6/6 2024-02-20T12:48:07.435 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-mgr-cephadm-2:16.2.14-548.g81bd20d6.el8.noarch 1/6 2024-02-20T12:48:07.435 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 2/6 2024-02-20T12:48:07.435 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-jinja2-2.10.1-4.el8.noarch 3/6 2024-02-20T12:48:07.435 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-apipkg-1.5-6.el8.noarch 4/6 2024-02-20T12:48:07.435 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-execnet-1.7.1-1.el8.noarch 5/6 2024-02-20T12:48:07.454 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: librados-devel-2:16.2.14-548.g81bd20d6.el8.x86_64 1/1 2024-02-20T12:48:07.838 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-remoto-1.2.1-1.el8.noarch 6/6 2024-02-20T12:48:07.838 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-20T12:48:07.838 INFO:teuthology.orchestra.run.smithi064.stdout:Installed: 2024-02-20T12:48:07.838 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mgr-cephadm-2:16.2.14-548.g81bd20d6.el8.noarch 2024-02-20T12:48:07.838 INFO:teuthology.orchestra.run.smithi064.stdout: python3-apipkg-1.5-6.el8.noarch 2024-02-20T12:48:07.838 INFO:teuthology.orchestra.run.smithi064.stdout: python3-babel-2.5.1-7.el8.noarch 2024-02-20T12:48:07.839 INFO:teuthology.orchestra.run.smithi064.stdout: python3-execnet-1.7.1-1.el8.noarch 2024-02-20T12:48:07.839 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jinja2-2.10.1-4.el8.noarch 2024-02-20T12:48:07.839 INFO:teuthology.orchestra.run.smithi064.stdout: python3-remoto-1.2.1-1.el8.noarch 2024-02-20T12:48:07.839 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-20T12:48:07.839 INFO:teuthology.orchestra.run.smithi064.stdout:Complete! 2024-02-20T12:48:07.973 DEBUG:teuthology.orchestra.run.smithi064:> sudo yum -y install ceph-fuse 2024-02-20T12:48:08.663 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : librados-devel-2:16.2.14-548.g81bd20d6.el8.x86_64 1/1 2024-02-20T12:48:08.663 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-20T12:48:08.663 INFO:teuthology.orchestra.run.smithi071.stdout:Installed: 2024-02-20T12:48:08.663 INFO:teuthology.orchestra.run.smithi071.stdout: librados-devel-2:16.2.14-548.g81bd20d6.el8.x86_64 2024-02-20T12:48:08.663 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-20T12:48:08.663 INFO:teuthology.orchestra.run.smithi071.stdout:Complete! 2024-02-20T12:48:08.763 INFO:teuthology.orchestra.run.smithi064.stdout:Last metadata expiration check: 0:02:14 ago on Tue 20 Feb 2024 12:45:54 PM UTC. 2024-02-20T12:48:08.885 DEBUG:teuthology.orchestra.run.smithi071:> sudo yum -y install libcephfs2 2024-02-20T12:48:09.655 INFO:teuthology.orchestra.run.smithi071.stdout:Last metadata expiration check: 0:02:18 ago on Tue 20 Feb 2024 12:45:51 PM UTC. 2024-02-20T12:48:10.472 INFO:teuthology.orchestra.run.smithi064.stdout:Dependencies resolved. 2024-02-20T12:48:10.473 INFO:teuthology.orchestra.run.smithi064.stdout:================================================================================ 2024-02-20T12:48:10.474 INFO:teuthology.orchestra.run.smithi064.stdout: Package Arch Version Repository Size 2024-02-20T12:48:10.474 INFO:teuthology.orchestra.run.smithi064.stdout:================================================================================ 2024-02-20T12:48:10.474 INFO:teuthology.orchestra.run.smithi064.stdout:Installing: 2024-02-20T12:48:10.474 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-fuse x86_64 2:16.2.14-548.g81bd20d6.el8 ceph 911 k 2024-02-20T12:48:10.474 INFO:teuthology.orchestra.run.smithi064.stdout:Installing dependencies: 2024-02-20T12:48:10.475 INFO:teuthology.orchestra.run.smithi064.stdout: fuse x86_64 2.9.7-17.el8 baseos 83 k 2024-02-20T12:48:10.475 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-20T12:48:10.475 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction Summary 2024-02-20T12:48:10.475 INFO:teuthology.orchestra.run.smithi064.stdout:================================================================================ 2024-02-20T12:48:10.475 INFO:teuthology.orchestra.run.smithi064.stdout:Install 2 Packages 2024-02-20T12:48:10.475 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-20T12:48:10.476 INFO:teuthology.orchestra.run.smithi064.stdout:Total download size: 994 k 2024-02-20T12:48:10.476 INFO:teuthology.orchestra.run.smithi064.stdout:Installed size: 2.8 M 2024-02-20T12:48:10.476 INFO:teuthology.orchestra.run.smithi064.stdout:Downloading Packages: 2024-02-20T12:48:11.060 INFO:teuthology.orchestra.run.smithi064.stdout:(1/2): fuse-2.9.7-17.el8.x86_64.rpm 1.2 MB/s | 83 kB 00:00 2024-02-20T12:48:11.174 INFO:teuthology.orchestra.run.smithi071.stdout:Package libcephfs2-2:16.2.14-548.g81bd20d6.el8.x86_64 is already installed. 2024-02-20T12:48:11.221 INFO:teuthology.orchestra.run.smithi071.stdout:Dependencies resolved. 2024-02-20T12:48:11.222 INFO:teuthology.orchestra.run.smithi071.stdout:Nothing to do. 2024-02-20T12:48:11.222 INFO:teuthology.orchestra.run.smithi071.stdout:Complete! 2024-02-20T12:48:11.317 DEBUG:teuthology.orchestra.run.smithi071:> sudo yum -y install libcephfs-devel 2024-02-20T12:48:11.579 INFO:teuthology.orchestra.run.smithi064.stdout:(2/2): ceph-fuse-16.2.14-548.g81bd20d6.el8.x86_ 1.5 MB/s | 911 kB 00:00 2024-02-20T12:48:11.579 INFO:teuthology.orchestra.run.smithi064.stdout:-------------------------------------------------------------------------------- 2024-02-20T12:48:11.580 INFO:teuthology.orchestra.run.smithi064.stdout:Total 899 kB/s | 994 kB 00:01 2024-02-20T12:48:11.592 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction check 2024-02-20T12:48:11.616 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction check succeeded. 2024-02-20T12:48:11.616 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction test 2024-02-20T12:48:11.718 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction test succeeded. 2024-02-20T12:48:11.724 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction 2024-02-20T12:48:11.946 INFO:teuthology.orchestra.run.smithi064.stdout: Preparing : 1/1 2024-02-20T12:48:12.076 INFO:teuthology.orchestra.run.smithi071.stdout:Last metadata expiration check: 0:02:21 ago on Tue 20 Feb 2024 12:45:51 PM UTC. 2024-02-20T12:48:12.258 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : fuse-2.9.7-17.el8.x86_64 1/2 2024-02-20T12:48:12.523 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-fuse-2:16.2.14-548.g81bd20d6.el8.x86_64 2/2 2024-02-20T12:48:12.939 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-fuse-2:16.2.14-548.g81bd20d6.el8.x86_64 2/2 2024-02-20T12:48:12.939 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-fuse-2:16.2.14-548.g81bd20d6.el8.x86_64 1/2 2024-02-20T12:48:13.322 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : fuse-2.9.7-17.el8.x86_64 2/2 2024-02-20T12:48:13.322 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-20T12:48:13.322 INFO:teuthology.orchestra.run.smithi064.stdout:Installed: 2024-02-20T12:48:13.323 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-fuse-2:16.2.14-548.g81bd20d6.el8.x86_64 fuse-2.9.7-17.el8.x86_64 2024-02-20T12:48:13.323 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-20T12:48:13.323 INFO:teuthology.orchestra.run.smithi064.stdout:Complete! 2024-02-20T12:48:13.565 DEBUG:teuthology.orchestra.run.smithi064:> sudo yum -y install librados-devel 2024-02-20T12:48:13.626 INFO:teuthology.orchestra.run.smithi071.stdout:Dependencies resolved. 2024-02-20T12:48:13.627 INFO:teuthology.orchestra.run.smithi071.stdout:================================================================================ 2024-02-20T12:48:13.627 INFO:teuthology.orchestra.run.smithi071.stdout: Package Arch Version Repo Size 2024-02-20T12:48:13.627 INFO:teuthology.orchestra.run.smithi071.stdout:================================================================================ 2024-02-20T12:48:13.627 INFO:teuthology.orchestra.run.smithi071.stdout:Installing: 2024-02-20T12:48:13.627 INFO:teuthology.orchestra.run.smithi071.stdout: libcephfs-devel x86_64 2:16.2.14-548.g81bd20d6.el8 ceph 25 k 2024-02-20T12:48:13.627 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-20T12:48:13.627 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction Summary 2024-02-20T12:48:13.627 INFO:teuthology.orchestra.run.smithi071.stdout:================================================================================ 2024-02-20T12:48:13.627 INFO:teuthology.orchestra.run.smithi071.stdout:Install 1 Package 2024-02-20T12:48:13.627 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-20T12:48:13.627 INFO:teuthology.orchestra.run.smithi071.stdout:Total download size: 25 k 2024-02-20T12:48:13.628 INFO:teuthology.orchestra.run.smithi071.stdout:Installed size: 107 k 2024-02-20T12:48:13.628 INFO:teuthology.orchestra.run.smithi071.stdout:Downloading Packages: 2024-02-20T12:48:14.201 INFO:teuthology.orchestra.run.smithi071.stdout:libcephfs-devel-16.2.14-548.g81bd20d6.el8.x86_6 45 kB/s | 25 kB 00:00 2024-02-20T12:48:14.201 INFO:teuthology.orchestra.run.smithi071.stdout:-------------------------------------------------------------------------------- 2024-02-20T12:48:14.202 INFO:teuthology.orchestra.run.smithi071.stdout:Total 44 kB/s | 25 kB 00:00 2024-02-20T12:48:14.202 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction check 2024-02-20T12:48:14.206 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction check succeeded. 2024-02-20T12:48:14.207 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction test 2024-02-20T12:48:14.209 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction test succeeded. 2024-02-20T12:48:14.213 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction 2024-02-20T12:48:14.366 INFO:teuthology.orchestra.run.smithi064.stdout:Last metadata expiration check: 0:02:20 ago on Tue 20 Feb 2024 12:45:54 PM UTC. 2024-02-20T12:48:14.445 INFO:teuthology.orchestra.run.smithi071.stdout: Preparing : 1/1 2024-02-20T12:48:14.660 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : libcephfs-devel-2:16.2.14-548.g81bd20d6.el8.x86_64 1/1 2024-02-20T12:48:14.982 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: libcephfs-devel-2:16.2.14-548.g81bd20d6.el8.x86_64 1/1 2024-02-20T12:48:15.312 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : libcephfs-devel-2:16.2.14-548.g81bd20d6.el8.x86_64 1/1 2024-02-20T12:48:15.312 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-20T12:48:15.312 INFO:teuthology.orchestra.run.smithi071.stdout:Installed: 2024-02-20T12:48:15.312 INFO:teuthology.orchestra.run.smithi071.stdout: libcephfs-devel-2:16.2.14-548.g81bd20d6.el8.x86_64 2024-02-20T12:48:15.313 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-20T12:48:15.313 INFO:teuthology.orchestra.run.smithi071.stdout:Complete! 2024-02-20T12:48:15.427 DEBUG:teuthology.orchestra.run.smithi071:> sudo yum -y install librados2 2024-02-20T12:48:16.053 INFO:teuthology.orchestra.run.smithi064.stdout:Dependencies resolved. 2024-02-20T12:48:16.053 INFO:teuthology.orchestra.run.smithi064.stdout:================================================================================ 2024-02-20T12:48:16.053 INFO:teuthology.orchestra.run.smithi064.stdout: Package Arch Version Repo Size 2024-02-20T12:48:16.054 INFO:teuthology.orchestra.run.smithi064.stdout:================================================================================ 2024-02-20T12:48:16.054 INFO:teuthology.orchestra.run.smithi064.stdout:Installing: 2024-02-20T12:48:16.054 INFO:teuthology.orchestra.run.smithi064.stdout: librados-devel x86_64 2:16.2.14-548.g81bd20d6.el8 ceph 131 k 2024-02-20T12:48:16.054 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-20T12:48:16.054 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction Summary 2024-02-20T12:48:16.054 INFO:teuthology.orchestra.run.smithi064.stdout:================================================================================ 2024-02-20T12:48:16.054 INFO:teuthology.orchestra.run.smithi064.stdout:Install 1 Package 2024-02-20T12:48:16.054 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-20T12:48:16.055 INFO:teuthology.orchestra.run.smithi064.stdout:Total download size: 131 k 2024-02-20T12:48:16.055 INFO:teuthology.orchestra.run.smithi064.stdout:Installed size: 432 k 2024-02-20T12:48:16.055 INFO:teuthology.orchestra.run.smithi064.stdout:Downloading Packages: 2024-02-20T12:48:16.181 INFO:teuthology.orchestra.run.smithi071.stdout:Last metadata expiration check: 0:02:25 ago on Tue 20 Feb 2024 12:45:51 PM UTC. 2024-02-20T12:48:16.375 INFO:teuthology.orchestra.run.smithi064.stdout:librados-devel-16.2.14-548.g81bd20d6.el8.x86_64 409 kB/s | 131 kB 00:00 2024-02-20T12:48:16.376 INFO:teuthology.orchestra.run.smithi064.stdout:-------------------------------------------------------------------------------- 2024-02-20T12:48:16.376 INFO:teuthology.orchestra.run.smithi064.stdout:Total 407 kB/s | 131 kB 00:00 2024-02-20T12:48:16.376 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction check 2024-02-20T12:48:16.383 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction check succeeded. 2024-02-20T12:48:16.383 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction test 2024-02-20T12:48:16.439 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction test succeeded. 2024-02-20T12:48:16.442 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction 2024-02-20T12:48:16.592 INFO:teuthology.orchestra.run.smithi064.stdout: Preparing : 1/1 2024-02-20T12:48:16.837 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : librados-devel-2:16.2.14-548.g81bd20d6.el8.x86_64 1/1 2024-02-20T12:48:17.126 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: librados-devel-2:16.2.14-548.g81bd20d6.el8.x86_64 1/1 2024-02-20T12:48:17.511 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : librados-devel-2:16.2.14-548.g81bd20d6.el8.x86_64 1/1 2024-02-20T12:48:17.511 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-20T12:48:17.511 INFO:teuthology.orchestra.run.smithi064.stdout:Installed: 2024-02-20T12:48:17.511 INFO:teuthology.orchestra.run.smithi064.stdout: librados-devel-2:16.2.14-548.g81bd20d6.el8.x86_64 2024-02-20T12:48:17.511 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-20T12:48:17.511 INFO:teuthology.orchestra.run.smithi064.stdout:Complete! 2024-02-20T12:48:17.669 INFO:teuthology.orchestra.run.smithi071.stdout:Package librados2-2:16.2.14-548.g81bd20d6.el8.x86_64 is already installed. 2024-02-20T12:48:17.715 INFO:teuthology.orchestra.run.smithi071.stdout:Dependencies resolved. 2024-02-20T12:48:17.716 INFO:teuthology.orchestra.run.smithi071.stdout:Nothing to do. 2024-02-20T12:48:17.716 INFO:teuthology.orchestra.run.smithi071.stdout:Complete! 2024-02-20T12:48:17.768 DEBUG:teuthology.orchestra.run.smithi064:> sudo yum -y install libcephfs2 2024-02-20T12:48:17.813 DEBUG:teuthology.orchestra.run.smithi071:> sudo yum -y install librbd1 2024-02-20T12:48:18.567 INFO:teuthology.orchestra.run.smithi064.stdout:Last metadata expiration check: 0:02:24 ago on Tue 20 Feb 2024 12:45:54 PM UTC. 2024-02-20T12:48:18.567 INFO:teuthology.orchestra.run.smithi071.stdout:Last metadata expiration check: 0:02:27 ago on Tue 20 Feb 2024 12:45:51 PM UTC. 2024-02-20T12:48:20.045 INFO:teuthology.orchestra.run.smithi071.stdout:Package librbd1-2:16.2.14-548.g81bd20d6.el8.x86_64 is already installed. 2024-02-20T12:48:20.091 INFO:teuthology.orchestra.run.smithi071.stdout:Dependencies resolved. 2024-02-20T12:48:20.092 INFO:teuthology.orchestra.run.smithi071.stdout:Nothing to do. 2024-02-20T12:48:20.093 INFO:teuthology.orchestra.run.smithi071.stdout:Complete! 2024-02-20T12:48:20.219 INFO:teuthology.orchestra.run.smithi064.stdout:Package libcephfs2-2:16.2.14-548.g81bd20d6.el8.x86_64 is already installed. 2024-02-20T12:48:20.269 INFO:teuthology.orchestra.run.smithi064.stdout:Dependencies resolved. 2024-02-20T12:48:20.270 INFO:teuthology.orchestra.run.smithi064.stdout:Nothing to do. 2024-02-20T12:48:20.271 INFO:teuthology.orchestra.run.smithi064.stdout:Complete! 2024-02-20T12:48:20.399 DEBUG:teuthology.orchestra.run.smithi064:> sudo yum -y install libcephfs-devel 2024-02-20T12:48:20.487 DEBUG:teuthology.orchestra.run.smithi071:> sudo yum -y install python3-rados 2024-02-20T12:48:21.194 INFO:teuthology.orchestra.run.smithi064.stdout:Last metadata expiration check: 0:02:27 ago on Tue 20 Feb 2024 12:45:54 PM UTC. 2024-02-20T12:48:21.234 INFO:teuthology.orchestra.run.smithi071.stdout:Last metadata expiration check: 0:02:30 ago on Tue 20 Feb 2024 12:45:51 PM UTC. 2024-02-20T12:48:22.734 INFO:teuthology.orchestra.run.smithi071.stdout:Package python3-rados-2:16.2.14-548.g81bd20d6.el8.x86_64 is already installed. 2024-02-20T12:48:22.780 INFO:teuthology.orchestra.run.smithi071.stdout:Dependencies resolved. 2024-02-20T12:48:22.781 INFO:teuthology.orchestra.run.smithi071.stdout:Nothing to do. 2024-02-20T12:48:22.782 INFO:teuthology.orchestra.run.smithi071.stdout:Complete! 2024-02-20T12:48:22.866 INFO:teuthology.orchestra.run.smithi064.stdout:Dependencies resolved. 2024-02-20T12:48:22.867 INFO:teuthology.orchestra.run.smithi064.stdout:================================================================================ 2024-02-20T12:48:22.867 INFO:teuthology.orchestra.run.smithi064.stdout: Package Arch Version Repo Size 2024-02-20T12:48:22.867 INFO:teuthology.orchestra.run.smithi064.stdout:================================================================================ 2024-02-20T12:48:22.867 INFO:teuthology.orchestra.run.smithi064.stdout:Installing: 2024-02-20T12:48:22.868 INFO:teuthology.orchestra.run.smithi064.stdout: libcephfs-devel x86_64 2:16.2.14-548.g81bd20d6.el8 ceph 25 k 2024-02-20T12:48:22.868 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-20T12:48:22.868 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction Summary 2024-02-20T12:48:22.868 INFO:teuthology.orchestra.run.smithi064.stdout:================================================================================ 2024-02-20T12:48:22.868 INFO:teuthology.orchestra.run.smithi064.stdout:Install 1 Package 2024-02-20T12:48:22.868 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-20T12:48:22.868 INFO:teuthology.orchestra.run.smithi064.stdout:Total download size: 25 k 2024-02-20T12:48:22.868 INFO:teuthology.orchestra.run.smithi064.stdout:Installed size: 107 k 2024-02-20T12:48:22.868 INFO:teuthology.orchestra.run.smithi064.stdout:Downloading Packages: 2024-02-20T12:48:22.878 DEBUG:teuthology.orchestra.run.smithi071:> sudo yum -y install python3-rgw 2024-02-20T12:48:23.197 INFO:teuthology.orchestra.run.smithi064.stdout:libcephfs-devel-16.2.14-548.g81bd20d6.el8.x86_6 77 kB/s | 25 kB 00:00 2024-02-20T12:48:23.198 INFO:teuthology.orchestra.run.smithi064.stdout:-------------------------------------------------------------------------------- 2024-02-20T12:48:23.198 INFO:teuthology.orchestra.run.smithi064.stdout:Total 77 kB/s | 25 kB 00:00 2024-02-20T12:48:23.198 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction check 2024-02-20T12:48:23.202 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction check succeeded. 2024-02-20T12:48:23.202 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction test 2024-02-20T12:48:23.205 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction test succeeded. 2024-02-20T12:48:23.208 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction 2024-02-20T12:48:23.262 INFO:teuthology.orchestra.run.smithi064.stdout: Preparing : 1/1 2024-02-20T12:48:23.511 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : libcephfs-devel-2:16.2.14-548.g81bd20d6.el8.x86_64 1/1 2024-02-20T12:48:23.636 INFO:teuthology.orchestra.run.smithi071.stdout:Last metadata expiration check: 0:02:32 ago on Tue 20 Feb 2024 12:45:51 PM UTC. 2024-02-20T12:48:23.798 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: libcephfs-devel-2:16.2.14-548.g81bd20d6.el8.x86_64 1/1 2024-02-20T12:48:24.228 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libcephfs-devel-2:16.2.14-548.g81bd20d6.el8.x86_64 1/1 2024-02-20T12:48:24.228 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-20T12:48:24.228 INFO:teuthology.orchestra.run.smithi064.stdout:Installed: 2024-02-20T12:48:24.228 INFO:teuthology.orchestra.run.smithi064.stdout: libcephfs-devel-2:16.2.14-548.g81bd20d6.el8.x86_64 2024-02-20T12:48:24.228 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-20T12:48:24.228 INFO:teuthology.orchestra.run.smithi064.stdout:Complete! 2024-02-20T12:48:24.355 DEBUG:teuthology.orchestra.run.smithi064:> sudo yum -y install librados2 2024-02-20T12:48:25.143 INFO:teuthology.orchestra.run.smithi064.stdout:Last metadata expiration check: 0:02:31 ago on Tue 20 Feb 2024 12:45:54 PM UTC. 2024-02-20T12:48:25.151 INFO:teuthology.orchestra.run.smithi071.stdout:Package python3-rgw-2:16.2.14-548.g81bd20d6.el8.x86_64 is already installed. 2024-02-20T12:48:25.198 INFO:teuthology.orchestra.run.smithi071.stdout:Dependencies resolved. 2024-02-20T12:48:25.199 INFO:teuthology.orchestra.run.smithi071.stdout:Nothing to do. 2024-02-20T12:48:25.199 INFO:teuthology.orchestra.run.smithi071.stdout:Complete! 2024-02-20T12:48:25.301 DEBUG:teuthology.orchestra.run.smithi071:> sudo yum -y install python3-cephfs 2024-02-20T12:48:26.059 INFO:teuthology.orchestra.run.smithi071.stdout:Last metadata expiration check: 0:02:35 ago on Tue 20 Feb 2024 12:45:51 PM UTC. 2024-02-20T12:48:26.786 INFO:teuthology.orchestra.run.smithi064.stdout:Package librados2-2:16.2.14-548.g81bd20d6.el8.x86_64 is already installed. 2024-02-20T12:48:26.836 INFO:teuthology.orchestra.run.smithi064.stdout:Dependencies resolved. 2024-02-20T12:48:26.837 INFO:teuthology.orchestra.run.smithi064.stdout:Nothing to do. 2024-02-20T12:48:26.837 INFO:teuthology.orchestra.run.smithi064.stdout:Complete! 2024-02-20T12:48:26.933 DEBUG:teuthology.orchestra.run.smithi064:> sudo yum -y install librbd1 2024-02-20T12:48:27.568 INFO:teuthology.orchestra.run.smithi071.stdout:Package python3-cephfs-2:16.2.14-548.g81bd20d6.el8.x86_64 is already installed. 2024-02-20T12:48:27.615 INFO:teuthology.orchestra.run.smithi071.stdout:Dependencies resolved. 2024-02-20T12:48:27.616 INFO:teuthology.orchestra.run.smithi071.stdout:Nothing to do. 2024-02-20T12:48:27.616 INFO:teuthology.orchestra.run.smithi071.stdout:Complete! 2024-02-20T12:48:27.718 DEBUG:teuthology.orchestra.run.smithi071:> sudo yum -y install python3-rbd 2024-02-20T12:48:27.725 INFO:teuthology.orchestra.run.smithi064.stdout:Last metadata expiration check: 0:02:33 ago on Tue 20 Feb 2024 12:45:54 PM UTC. 2024-02-20T12:48:28.465 INFO:teuthology.orchestra.run.smithi071.stdout:Last metadata expiration check: 0:02:37 ago on Tue 20 Feb 2024 12:45:51 PM UTC. 2024-02-20T12:48:29.337 INFO:teuthology.orchestra.run.smithi064.stdout:Package librbd1-2:16.2.14-548.g81bd20d6.el8.x86_64 is already installed. 2024-02-20T12:48:29.391 INFO:teuthology.orchestra.run.smithi064.stdout:Dependencies resolved. 2024-02-20T12:48:29.393 INFO:teuthology.orchestra.run.smithi064.stdout:Nothing to do. 2024-02-20T12:48:29.393 INFO:teuthology.orchestra.run.smithi064.stdout:Complete! 2024-02-20T12:48:29.500 DEBUG:teuthology.orchestra.run.smithi064:> sudo yum -y install python3-rados 2024-02-20T12:48:29.971 INFO:teuthology.orchestra.run.smithi071.stdout:Package python3-rbd-2:16.2.14-548.g81bd20d6.el8.x86_64 is already installed. 2024-02-20T12:48:30.018 INFO:teuthology.orchestra.run.smithi071.stdout:Dependencies resolved. 2024-02-20T12:48:30.019 INFO:teuthology.orchestra.run.smithi071.stdout:Nothing to do. 2024-02-20T12:48:30.019 INFO:teuthology.orchestra.run.smithi071.stdout:Complete! 2024-02-20T12:48:30.115 DEBUG:teuthology.orchestra.run.smithi071:> sudo yum -y install rbd-fuse 2024-02-20T12:48:30.284 INFO:teuthology.orchestra.run.smithi064.stdout:Last metadata expiration check: 0:02:36 ago on Tue 20 Feb 2024 12:45:54 PM UTC. 2024-02-20T12:48:30.871 INFO:teuthology.orchestra.run.smithi071.stdout:Last metadata expiration check: 0:02:39 ago on Tue 20 Feb 2024 12:45:51 PM UTC. 2024-02-20T12:48:31.908 INFO:teuthology.orchestra.run.smithi064.stdout:Package python3-rados-2:16.2.14-548.g81bd20d6.el8.x86_64 is already installed. 2024-02-20T12:48:31.958 INFO:teuthology.orchestra.run.smithi064.stdout:Dependencies resolved. 2024-02-20T12:48:31.959 INFO:teuthology.orchestra.run.smithi064.stdout:Nothing to do. 2024-02-20T12:48:31.960 INFO:teuthology.orchestra.run.smithi064.stdout:Complete! 2024-02-20T12:48:32.083 DEBUG:teuthology.orchestra.run.smithi064:> sudo yum -y install python3-rgw 2024-02-20T12:48:32.423 INFO:teuthology.orchestra.run.smithi071.stdout:Dependencies resolved. 2024-02-20T12:48:32.424 INFO:teuthology.orchestra.run.smithi071.stdout:================================================================================ 2024-02-20T12:48:32.424 INFO:teuthology.orchestra.run.smithi071.stdout: Package Architecture Version Repository Size 2024-02-20T12:48:32.424 INFO:teuthology.orchestra.run.smithi071.stdout:================================================================================ 2024-02-20T12:48:32.424 INFO:teuthology.orchestra.run.smithi071.stdout:Installing: 2024-02-20T12:48:32.424 INFO:teuthology.orchestra.run.smithi071.stdout: rbd-fuse x86_64 2:16.2.14-548.g81bd20d6.el8 ceph 88 k 2024-02-20T12:48:32.425 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-20T12:48:32.425 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction Summary 2024-02-20T12:48:32.425 INFO:teuthology.orchestra.run.smithi071.stdout:================================================================================ 2024-02-20T12:48:32.425 INFO:teuthology.orchestra.run.smithi071.stdout:Install 1 Package 2024-02-20T12:48:32.425 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-20T12:48:32.425 INFO:teuthology.orchestra.run.smithi071.stdout:Total download size: 88 k 2024-02-20T12:48:32.425 INFO:teuthology.orchestra.run.smithi071.stdout:Installed size: 240 k 2024-02-20T12:48:32.425 INFO:teuthology.orchestra.run.smithi071.stdout:Downloading Packages: 2024-02-20T12:48:32.601 INFO:teuthology.orchestra.run.smithi071.stdout:rbd-fuse-16.2.14-548.g81bd20d6.el8.x86_64.rpm 502 kB/s | 88 kB 00:00 2024-02-20T12:48:32.601 INFO:teuthology.orchestra.run.smithi071.stdout:-------------------------------------------------------------------------------- 2024-02-20T12:48:32.602 INFO:teuthology.orchestra.run.smithi071.stdout:Total 499 kB/s | 88 kB 00:00 2024-02-20T12:48:32.602 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction check 2024-02-20T12:48:32.611 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction check succeeded. 2024-02-20T12:48:32.611 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction test 2024-02-20T12:48:32.661 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction test succeeded. 2024-02-20T12:48:32.664 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction 2024-02-20T12:48:32.766 INFO:teuthology.orchestra.run.smithi071.stdout: Preparing : 1/1 2024-02-20T12:48:32.868 INFO:teuthology.orchestra.run.smithi064.stdout:Last metadata expiration check: 0:02:38 ago on Tue 20 Feb 2024 12:45:54 PM UTC. 2024-02-20T12:48:32.981 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : rbd-fuse-2:16.2.14-548.g81bd20d6.el8.x86_64 1/1 2024-02-20T12:48:33.272 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: rbd-fuse-2:16.2.14-548.g81bd20d6.el8.x86_64 1/1 2024-02-20T12:48:33.631 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : rbd-fuse-2:16.2.14-548.g81bd20d6.el8.x86_64 1/1 2024-02-20T12:48:33.632 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-20T12:48:33.632 INFO:teuthology.orchestra.run.smithi071.stdout:Installed: 2024-02-20T12:48:33.632 INFO:teuthology.orchestra.run.smithi071.stdout: rbd-fuse-2:16.2.14-548.g81bd20d6.el8.x86_64 2024-02-20T12:48:33.632 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-20T12:48:33.632 INFO:teuthology.orchestra.run.smithi071.stdout:Complete! 2024-02-20T12:48:33.840 DEBUG:teuthology.orchestra.run.smithi071:> sudo yum -y install rbd-mirror 2024-02-20T12:48:34.478 INFO:teuthology.orchestra.run.smithi064.stdout:Package python3-rgw-2:16.2.14-548.g81bd20d6.el8.x86_64 is already installed. 2024-02-20T12:48:34.528 INFO:teuthology.orchestra.run.smithi064.stdout:Dependencies resolved. 2024-02-20T12:48:34.529 INFO:teuthology.orchestra.run.smithi064.stdout:Nothing to do. 2024-02-20T12:48:34.529 INFO:teuthology.orchestra.run.smithi064.stdout:Complete! 2024-02-20T12:48:34.594 INFO:teuthology.orchestra.run.smithi071.stdout:Last metadata expiration check: 0:02:43 ago on Tue 20 Feb 2024 12:45:51 PM UTC. 2024-02-20T12:48:34.624 DEBUG:teuthology.orchestra.run.smithi064:> sudo yum -y install python3-cephfs 2024-02-20T12:48:35.419 INFO:teuthology.orchestra.run.smithi064.stdout:Last metadata expiration check: 0:02:41 ago on Tue 20 Feb 2024 12:45:54 PM UTC. 2024-02-20T12:48:36.147 INFO:teuthology.orchestra.run.smithi071.stdout:Dependencies resolved. 2024-02-20T12:48:36.148 INFO:teuthology.orchestra.run.smithi071.stdout:================================================================================ 2024-02-20T12:48:36.148 INFO:teuthology.orchestra.run.smithi071.stdout: Package Arch Version Repo Size 2024-02-20T12:48:36.148 INFO:teuthology.orchestra.run.smithi071.stdout:================================================================================ 2024-02-20T12:48:36.148 INFO:teuthology.orchestra.run.smithi071.stdout:Installing: 2024-02-20T12:48:36.148 INFO:teuthology.orchestra.run.smithi071.stdout: rbd-mirror x86_64 2:16.2.14-548.g81bd20d6.el8 ceph 3.7 M 2024-02-20T12:48:36.148 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-20T12:48:36.148 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction Summary 2024-02-20T12:48:36.148 INFO:teuthology.orchestra.run.smithi071.stdout:================================================================================ 2024-02-20T12:48:36.148 INFO:teuthology.orchestra.run.smithi071.stdout:Install 1 Package 2024-02-20T12:48:36.148 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-20T12:48:36.148 INFO:teuthology.orchestra.run.smithi071.stdout:Total download size: 3.7 M 2024-02-20T12:48:36.148 INFO:teuthology.orchestra.run.smithi071.stdout:Installed size: 15 M 2024-02-20T12:48:36.149 INFO:teuthology.orchestra.run.smithi071.stdout:Downloading Packages: 2024-02-20T12:48:36.853 INFO:teuthology.orchestra.run.smithi071.stdout:rbd-mirror-16.2.14-548.g81bd20d6.el8.x86_64.rpm 5.2 MB/s | 3.7 MB 00:00 2024-02-20T12:48:36.854 INFO:teuthology.orchestra.run.smithi071.stdout:-------------------------------------------------------------------------------- 2024-02-20T12:48:36.854 INFO:teuthology.orchestra.run.smithi071.stdout:Total 5.2 MB/s | 3.7 MB 00:00 2024-02-20T12:48:36.854 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction check 2024-02-20T12:48:36.866 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction check succeeded. 2024-02-20T12:48:36.866 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction test 2024-02-20T12:48:36.934 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction test succeeded. 2024-02-20T12:48:36.937 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction 2024-02-20T12:48:37.025 INFO:teuthology.orchestra.run.smithi064.stdout:Package python3-cephfs-2:16.2.14-548.g81bd20d6.el8.x86_64 is already installed. 2024-02-20T12:48:37.075 INFO:teuthology.orchestra.run.smithi064.stdout:Dependencies resolved. 2024-02-20T12:48:37.076 INFO:teuthology.orchestra.run.smithi064.stdout:Nothing to do. 2024-02-20T12:48:37.076 INFO:teuthology.orchestra.run.smithi064.stdout:Complete! 2024-02-20T12:48:37.173 DEBUG:teuthology.orchestra.run.smithi064:> sudo yum -y install python3-rbd 2024-02-20T12:48:37.373 INFO:teuthology.orchestra.run.smithi071.stdout: Preparing : 1/1 2024-02-20T12:48:37.548 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : rbd-mirror-2:16.2.14-548.g81bd20d6.el8.x86_64 1/1 2024-02-20T12:48:37.958 INFO:teuthology.orchestra.run.smithi064.stdout:Last metadata expiration check: 0:02:43 ago on Tue 20 Feb 2024 12:45:54 PM UTC. 2024-02-20T12:48:37.994 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: rbd-mirror-2:16.2.14-548.g81bd20d6.el8.x86_64 1/1 2024-02-20T12:48:38.419 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : rbd-mirror-2:16.2.14-548.g81bd20d6.el8.x86_64 1/1 2024-02-20T12:48:38.419 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-20T12:48:38.419 INFO:teuthology.orchestra.run.smithi071.stdout:Installed: 2024-02-20T12:48:38.419 INFO:teuthology.orchestra.run.smithi071.stdout: rbd-mirror-2:16.2.14-548.g81bd20d6.el8.x86_64 2024-02-20T12:48:38.419 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-20T12:48:38.419 INFO:teuthology.orchestra.run.smithi071.stdout:Complete! 2024-02-20T12:48:38.647 DEBUG:teuthology.orchestra.run.smithi071:> sudo yum -y install rbd-nbd 2024-02-20T12:48:39.409 INFO:teuthology.orchestra.run.smithi071.stdout:Last metadata expiration check: 0:02:48 ago on Tue 20 Feb 2024 12:45:51 PM UTC. 2024-02-20T12:48:39.583 INFO:teuthology.orchestra.run.smithi064.stdout:Package python3-rbd-2:16.2.14-548.g81bd20d6.el8.x86_64 is already installed. 2024-02-20T12:48:39.633 INFO:teuthology.orchestra.run.smithi064.stdout:Dependencies resolved. 2024-02-20T12:48:39.634 INFO:teuthology.orchestra.run.smithi064.stdout:Nothing to do. 2024-02-20T12:48:39.634 INFO:teuthology.orchestra.run.smithi064.stdout:Complete! 2024-02-20T12:48:39.727 DEBUG:teuthology.orchestra.run.smithi064:> sudo yum -y install rbd-fuse 2024-02-20T12:48:40.520 INFO:teuthology.orchestra.run.smithi064.stdout:Last metadata expiration check: 0:02:46 ago on Tue 20 Feb 2024 12:45:54 PM UTC. 2024-02-20T12:48:40.972 INFO:teuthology.orchestra.run.smithi071.stdout:Dependencies resolved. 2024-02-20T12:48:40.973 INFO:teuthology.orchestra.run.smithi071.stdout:================================================================================ 2024-02-20T12:48:40.973 INFO:teuthology.orchestra.run.smithi071.stdout: Package Architecture Version Repository Size 2024-02-20T12:48:40.973 INFO:teuthology.orchestra.run.smithi071.stdout:================================================================================ 2024-02-20T12:48:40.973 INFO:teuthology.orchestra.run.smithi071.stdout:Installing: 2024-02-20T12:48:40.973 INFO:teuthology.orchestra.run.smithi071.stdout: rbd-nbd x86_64 2:16.2.14-548.g81bd20d6.el8 ceph 175 k 2024-02-20T12:48:40.973 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-20T12:48:40.974 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction Summary 2024-02-20T12:48:40.974 INFO:teuthology.orchestra.run.smithi071.stdout:================================================================================ 2024-02-20T12:48:40.974 INFO:teuthology.orchestra.run.smithi071.stdout:Install 1 Package 2024-02-20T12:48:40.974 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-20T12:48:40.974 INFO:teuthology.orchestra.run.smithi071.stdout:Total download size: 175 k 2024-02-20T12:48:40.974 INFO:teuthology.orchestra.run.smithi071.stdout:Installed size: 523 k 2024-02-20T12:48:40.974 INFO:teuthology.orchestra.run.smithi071.stdout:Downloading Packages: 2024-02-20T12:48:41.150 INFO:teuthology.orchestra.run.smithi071.stdout:rbd-nbd-16.2.14-548.g81bd20d6.el8.x86_64.rpm 998 kB/s | 175 kB 00:00 2024-02-20T12:48:41.150 INFO:teuthology.orchestra.run.smithi071.stdout:-------------------------------------------------------------------------------- 2024-02-20T12:48:41.150 INFO:teuthology.orchestra.run.smithi071.stdout:Total 991 kB/s | 175 kB 00:00 2024-02-20T12:48:41.150 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction check 2024-02-20T12:48:41.161 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction check succeeded. 2024-02-20T12:48:41.161 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction test 2024-02-20T12:48:41.210 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction test succeeded. 2024-02-20T12:48:41.213 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction 2024-02-20T12:48:41.356 INFO:teuthology.orchestra.run.smithi071.stdout: Preparing : 1/1 2024-02-20T12:48:41.570 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : rbd-nbd-2:16.2.14-548.g81bd20d6.el8.x86_64 1/1 2024-02-20T12:48:41.918 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: rbd-nbd-2:16.2.14-548.g81bd20d6.el8.x86_64 1/1 2024-02-20T12:48:42.184 INFO:teuthology.orchestra.run.smithi064.stdout:Dependencies resolved. 2024-02-20T12:48:42.185 INFO:teuthology.orchestra.run.smithi064.stdout:================================================================================ 2024-02-20T12:48:42.185 INFO:teuthology.orchestra.run.smithi064.stdout: Package Architecture Version Repository Size 2024-02-20T12:48:42.185 INFO:teuthology.orchestra.run.smithi064.stdout:================================================================================ 2024-02-20T12:48:42.185 INFO:teuthology.orchestra.run.smithi064.stdout:Installing: 2024-02-20T12:48:42.185 INFO:teuthology.orchestra.run.smithi064.stdout: rbd-fuse x86_64 2:16.2.14-548.g81bd20d6.el8 ceph 88 k 2024-02-20T12:48:42.185 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-20T12:48:42.185 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction Summary 2024-02-20T12:48:42.185 INFO:teuthology.orchestra.run.smithi064.stdout:================================================================================ 2024-02-20T12:48:42.185 INFO:teuthology.orchestra.run.smithi064.stdout:Install 1 Package 2024-02-20T12:48:42.185 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-20T12:48:42.185 INFO:teuthology.orchestra.run.smithi064.stdout:Total download size: 88 k 2024-02-20T12:48:42.185 INFO:teuthology.orchestra.run.smithi064.stdout:Installed size: 240 k 2024-02-20T12:48:42.186 INFO:teuthology.orchestra.run.smithi064.stdout:Downloading Packages: 2024-02-20T12:48:42.285 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : rbd-nbd-2:16.2.14-548.g81bd20d6.el8.x86_64 1/1 2024-02-20T12:48:42.285 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-20T12:48:42.285 INFO:teuthology.orchestra.run.smithi071.stdout:Installed: 2024-02-20T12:48:42.285 INFO:teuthology.orchestra.run.smithi071.stdout: rbd-nbd-2:16.2.14-548.g81bd20d6.el8.x86_64 2024-02-20T12:48:42.285 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-20T12:48:42.285 INFO:teuthology.orchestra.run.smithi071.stdout:Complete! 2024-02-20T12:48:42.345 INFO:teuthology.orchestra.run.smithi064.stdout:rbd-fuse-16.2.14-548.g81bd20d6.el8.x86_64.rpm 554 kB/s | 88 kB 00:00 2024-02-20T12:48:42.346 INFO:teuthology.orchestra.run.smithi064.stdout:-------------------------------------------------------------------------------- 2024-02-20T12:48:42.346 INFO:teuthology.orchestra.run.smithi064.stdout:Total 550 kB/s | 88 kB 00:00 2024-02-20T12:48:42.346 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction check 2024-02-20T12:48:42.357 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction check succeeded. 2024-02-20T12:48:42.357 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction test 2024-02-20T12:48:42.410 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction test succeeded. 2024-02-20T12:48:42.414 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction 2024-02-20T12:48:42.490 DEBUG:teuthology.parallel:result is None 2024-02-20T12:48:42.556 INFO:teuthology.orchestra.run.smithi064.stdout: Preparing : 1/1 2024-02-20T12:48:42.767 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : rbd-fuse-2:16.2.14-548.g81bd20d6.el8.x86_64 1/1 2024-02-20T12:48:43.066 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: rbd-fuse-2:16.2.14-548.g81bd20d6.el8.x86_64 1/1 2024-02-20T12:48:43.468 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : rbd-fuse-2:16.2.14-548.g81bd20d6.el8.x86_64 1/1 2024-02-20T12:48:43.468 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-20T12:48:43.468 INFO:teuthology.orchestra.run.smithi064.stdout:Installed: 2024-02-20T12:48:43.468 INFO:teuthology.orchestra.run.smithi064.stdout: rbd-fuse-2:16.2.14-548.g81bd20d6.el8.x86_64 2024-02-20T12:48:43.468 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-20T12:48:43.468 INFO:teuthology.orchestra.run.smithi064.stdout:Complete! 2024-02-20T12:48:43.686 DEBUG:teuthology.orchestra.run.smithi064:> sudo yum -y install rbd-mirror 2024-02-20T12:48:44.470 INFO:teuthology.orchestra.run.smithi064.stdout:Last metadata expiration check: 0:02:50 ago on Tue 20 Feb 2024 12:45:54 PM UTC. 2024-02-20T12:48:46.147 INFO:teuthology.orchestra.run.smithi064.stdout:Dependencies resolved. 2024-02-20T12:48:46.148 INFO:teuthology.orchestra.run.smithi064.stdout:================================================================================ 2024-02-20T12:48:46.148 INFO:teuthology.orchestra.run.smithi064.stdout: Package Arch Version Repo Size 2024-02-20T12:48:46.148 INFO:teuthology.orchestra.run.smithi064.stdout:================================================================================ 2024-02-20T12:48:46.148 INFO:teuthology.orchestra.run.smithi064.stdout:Installing: 2024-02-20T12:48:46.148 INFO:teuthology.orchestra.run.smithi064.stdout: rbd-mirror x86_64 2:16.2.14-548.g81bd20d6.el8 ceph 3.7 M 2024-02-20T12:48:46.148 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-20T12:48:46.148 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction Summary 2024-02-20T12:48:46.148 INFO:teuthology.orchestra.run.smithi064.stdout:================================================================================ 2024-02-20T12:48:46.148 INFO:teuthology.orchestra.run.smithi064.stdout:Install 1 Package 2024-02-20T12:48:46.148 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-20T12:48:46.149 INFO:teuthology.orchestra.run.smithi064.stdout:Total download size: 3.7 M 2024-02-20T12:48:46.149 INFO:teuthology.orchestra.run.smithi064.stdout:Installed size: 15 M 2024-02-20T12:48:46.149 INFO:teuthology.orchestra.run.smithi064.stdout:Downloading Packages: 2024-02-20T12:48:47.207 INFO:teuthology.orchestra.run.smithi064.stdout:rbd-mirror-16.2.14-548.g81bd20d6.el8.x86_64.rpm 3.5 MB/s | 3.7 MB 00:01 2024-02-20T12:48:47.207 INFO:teuthology.orchestra.run.smithi064.stdout:-------------------------------------------------------------------------------- 2024-02-20T12:48:47.208 INFO:teuthology.orchestra.run.smithi064.stdout:Total 3.5 MB/s | 3.7 MB 00:01 2024-02-20T12:48:47.208 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction check 2024-02-20T12:48:47.222 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction check succeeded. 2024-02-20T12:48:47.222 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction test 2024-02-20T12:48:47.292 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction test succeeded. 2024-02-20T12:48:47.295 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction 2024-02-20T12:48:47.813 INFO:teuthology.orchestra.run.smithi064.stdout: Preparing : 1/1 2024-02-20T12:48:48.052 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : rbd-mirror-2:16.2.14-548.g81bd20d6.el8.x86_64 1/1 2024-02-20T12:48:48.530 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: rbd-mirror-2:16.2.14-548.g81bd20d6.el8.x86_64 1/1 2024-02-20T12:48:48.943 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : rbd-mirror-2:16.2.14-548.g81bd20d6.el8.x86_64 1/1 2024-02-20T12:48:48.943 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-20T12:48:48.943 INFO:teuthology.orchestra.run.smithi064.stdout:Installed: 2024-02-20T12:48:48.943 INFO:teuthology.orchestra.run.smithi064.stdout: rbd-mirror-2:16.2.14-548.g81bd20d6.el8.x86_64 2024-02-20T12:48:48.943 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-20T12:48:48.943 INFO:teuthology.orchestra.run.smithi064.stdout:Complete! 2024-02-20T12:48:49.202 DEBUG:teuthology.orchestra.run.smithi064:> sudo yum -y install rbd-nbd 2024-02-20T12:48:49.998 INFO:teuthology.orchestra.run.smithi064.stdout:Last metadata expiration check: 0:02:55 ago on Tue 20 Feb 2024 12:45:54 PM UTC. 2024-02-20T12:48:51.655 INFO:teuthology.orchestra.run.smithi064.stdout:Dependencies resolved. 2024-02-20T12:48:51.655 INFO:teuthology.orchestra.run.smithi064.stdout:================================================================================ 2024-02-20T12:48:51.655 INFO:teuthology.orchestra.run.smithi064.stdout: Package Architecture Version Repository Size 2024-02-20T12:48:51.655 INFO:teuthology.orchestra.run.smithi064.stdout:================================================================================ 2024-02-20T12:48:51.655 INFO:teuthology.orchestra.run.smithi064.stdout:Installing: 2024-02-20T12:48:51.655 INFO:teuthology.orchestra.run.smithi064.stdout: rbd-nbd x86_64 2:16.2.14-548.g81bd20d6.el8 ceph 175 k 2024-02-20T12:48:51.655 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-20T12:48:51.656 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction Summary 2024-02-20T12:48:51.656 INFO:teuthology.orchestra.run.smithi064.stdout:================================================================================ 2024-02-20T12:48:51.656 INFO:teuthology.orchestra.run.smithi064.stdout:Install 1 Package 2024-02-20T12:48:51.656 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-20T12:48:51.656 INFO:teuthology.orchestra.run.smithi064.stdout:Total download size: 175 k 2024-02-20T12:48:51.656 INFO:teuthology.orchestra.run.smithi064.stdout:Installed size: 523 k 2024-02-20T12:48:51.657 INFO:teuthology.orchestra.run.smithi064.stdout:Downloading Packages: 2024-02-20T12:48:51.852 INFO:teuthology.orchestra.run.smithi064.stdout:rbd-nbd-16.2.14-548.g81bd20d6.el8.x86_64.rpm 895 kB/s | 175 kB 00:00 2024-02-20T12:48:51.853 INFO:teuthology.orchestra.run.smithi064.stdout:-------------------------------------------------------------------------------- 2024-02-20T12:48:51.853 INFO:teuthology.orchestra.run.smithi064.stdout:Total 889 kB/s | 175 kB 00:00 2024-02-20T12:48:51.853 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction check 2024-02-20T12:48:51.865 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction check succeeded. 2024-02-20T12:48:51.865 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction test 2024-02-20T12:48:51.920 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction test succeeded. 2024-02-20T12:48:51.923 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction 2024-02-20T12:48:52.074 INFO:teuthology.orchestra.run.smithi064.stdout: Preparing : 1/1 2024-02-20T12:48:52.340 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : rbd-nbd-2:16.2.14-548.g81bd20d6.el8.x86_64 1/1 2024-02-20T12:48:52.684 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: rbd-nbd-2:16.2.14-548.g81bd20d6.el8.x86_64 1/1 2024-02-20T12:48:53.121 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : rbd-nbd-2:16.2.14-548.g81bd20d6.el8.x86_64 1/1 2024-02-20T12:48:53.121 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-20T12:48:53.121 INFO:teuthology.orchestra.run.smithi064.stdout:Installed: 2024-02-20T12:48:53.121 INFO:teuthology.orchestra.run.smithi064.stdout: rbd-nbd-2:16.2.14-548.g81bd20d6.el8.x86_64 2024-02-20T12:48:53.122 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-20T12:48:53.122 INFO:teuthology.orchestra.run.smithi064.stdout:Complete! 2024-02-20T12:48:53.355 DEBUG:teuthology.parallel:result is None 2024-02-20T12:48:53.356 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-02-20T12:48:53.356 INFO:teuthology.packaging:ref: None 2024-02-20T12:48:53.356 INFO:teuthology.packaging:tag: None 2024-02-20T12:48:53.356 INFO:teuthology.packaging:branch: pacific 2024-02-20T12:48:53.356 INFO:teuthology.packaging:sha1: b0bd01b8442d66b8a31f8cb3050334b026de2f3b 2024-02-20T12:48:53.356 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=pacific 2024-02-20T12:48:53.798 DEBUG:teuthology.orchestra.run.smithi064:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-02-20T12:48:53.823 INFO:teuthology.orchestra.run.smithi064.stdout:16.2.14-548.g81bd20d6.el8 2024-02-20T12:48:53.824 INFO:teuthology.packaging:The installed version of ceph is 16.2.14-548.g81bd20d6.el8 2024-02-20T12:48:53.824 INFO:teuthology.task.install:The correct ceph version 16.2.14-548.g81bd20d6 is installed. 2024-02-20T12:48:53.825 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-02-20T12:48:53.826 INFO:teuthology.packaging:ref: None 2024-02-20T12:48:53.826 INFO:teuthology.packaging:tag: None 2024-02-20T12:48:53.826 INFO:teuthology.packaging:branch: pacific 2024-02-20T12:48:53.826 INFO:teuthology.packaging:sha1: b0bd01b8442d66b8a31f8cb3050334b026de2f3b 2024-02-20T12:48:53.826 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=pacific 2024-02-20T12:48:54.198 DEBUG:teuthology.orchestra.run.smithi071:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-02-20T12:48:54.226 INFO:teuthology.orchestra.run.smithi071.stdout:16.2.14-548.g81bd20d6.el8 2024-02-20T12:48:54.226 INFO:teuthology.packaging:The installed version of ceph is 16.2.14-548.g81bd20d6.el8 2024-02-20T12:48:54.226 INFO:teuthology.task.install:The correct ceph version 16.2.14-548.g81bd20d6 is installed. 2024-02-20T12:48:54.228 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-02-20T12:48:54.228 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-02-20T12:48:54.228 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-02-20T12:48:54.259 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-02-20T12:48:54.260 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-02-20T12:48:54.298 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-02-20T12:48:54.298 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-02-20T12:48:54.298 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd of=/usr/bin/daemon-helper 2024-02-20T12:48:54.329 DEBUG:teuthology.orchestra.run.smithi064:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-02-20T12:48:54.398 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-02-20T12:48:54.398 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd of=/usr/bin/daemon-helper 2024-02-20T12:48:54.428 DEBUG:teuthology.orchestra.run.smithi071:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-02-20T12:48:54.497 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-02-20T12:48:54.498 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-02-20T12:48:54.498 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd of=/usr/bin/adjust-ulimits 2024-02-20T12:48:54.526 DEBUG:teuthology.orchestra.run.smithi064:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-02-20T12:48:54.594 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-02-20T12:48:54.594 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd of=/usr/bin/adjust-ulimits 2024-02-20T12:48:54.623 DEBUG:teuthology.orchestra.run.smithi071:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-02-20T12:48:54.691 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-02-20T12:48:54.691 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-02-20T12:48:54.691 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd of=/usr/bin/stdin-killer 2024-02-20T12:48:54.719 DEBUG:teuthology.orchestra.run.smithi064:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-02-20T12:48:54.787 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-02-20T12:48:54.787 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd of=/usr/bin/stdin-killer 2024-02-20T12:48:54.816 DEBUG:teuthology.orchestra.run.smithi071:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-02-20T12:48:54.885 INFO:teuthology.run_tasks:Running task print... 2024-02-20T12:48:54.895 INFO:teuthology.task.print:**** done install task... 2024-02-20T12:48:54.895 INFO:teuthology.run_tasks:Running task cephadm... 2024-02-20T12:48:55.002 INFO:tasks.cephadm:Config: {'cephadm_branch': 'pacific', 'cephadm_git_url': 'https://github.com/ceph/ceph', 'conf': {'osd': {'osd_class_default_list': '*', 'osd_class_load_list': '*', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'mon pg warn min per osd': 0}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}}, 'image': 'docker.io/ceph/daemon-base:latest-pacific', 'roleless': True, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding'], 'sha1': 'b0bd01b8442d66b8a31f8cb3050334b026de2f3b'} 2024-02-20T12:48:55.003 INFO:tasks.cephadm:Cluster image is docker.io/ceph/daemon-base:latest-pacific 2024-02-20T12:48:55.003 INFO:tasks.cephadm:Cluster fsid is 680ee126-cfee-11ee-95bb-87774f69a715 2024-02-20T12:48:55.003 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-02-20T12:48:55.003 INFO:tasks.cephadm:No mon roles; fabricating mons 2024-02-20T12:48:55.004 INFO:tasks.cephadm:Monitor IPs: {'mon.smithi064': '172.21.15.64', 'mon.smithi071': '172.21.15.71'} 2024-02-20T12:48:55.004 INFO:tasks.cephadm:Normalizing hostnames... 2024-02-20T12:48:55.004 DEBUG:teuthology.orchestra.run.smithi064:> sudo hostname $(hostname -s) 2024-02-20T12:48:55.033 DEBUG:teuthology.orchestra.run.smithi071:> sudo hostname $(hostname -s) 2024-02-20T12:48:55.064 INFO:tasks.cephadm:Downloading cephadm (repo https://github.com/ceph/ceph ref pacific)... 2024-02-20T12:48:55.064 DEBUG:teuthology.orchestra.run.smithi064:> curl --silent https://raw.githubusercontent.com/ceph/ceph/pacific/src/cephadm/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-02-20T12:48:55.362 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-rw-r--. 1 ubuntu ubuntu 355729 Feb 20 12:48 /home/ubuntu/cephtest/cephadm 2024-02-20T12:48:55.362 DEBUG:teuthology.orchestra.run.smithi071:> curl --silent https://raw.githubusercontent.com/ceph/ceph/pacific/src/cephadm/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-02-20T12:48:55.591 INFO:teuthology.orchestra.run.smithi071.stdout:-rw-rw-r--. 1 ubuntu ubuntu 355729 Feb 20 12:48 /home/ubuntu/cephtest/cephadm 2024-02-20T12:48:55.592 DEBUG:teuthology.orchestra.run.smithi064:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-02-20T12:48:55.612 DEBUG:teuthology.orchestra.run.smithi071:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-02-20T12:48:55.665 INFO:tasks.cephadm:Pulling image docker.io/ceph/daemon-base:latest-pacific on all hosts... 2024-02-20T12:48:55.665 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/daemon-base:latest-pacific pull 2024-02-20T12:48:55.667 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/daemon-base:latest-pacific pull 2024-02-20T12:48:56.757 INFO:teuthology.orchestra.run.smithi071.stderr:Pulling container image docker.io/ceph/daemon-base:latest-pacific... 2024-02-20T12:48:57.013 INFO:teuthology.orchestra.run.smithi064.stderr:Pulling container image docker.io/ceph/daemon-base:latest-pacific... 2024-02-20T12:49:27.138 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-02-20T12:49:27.139 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph_version": "ceph version 16.2.5 (0883bdea7337b95e4b611c768c0279868462204a) pacific (stable)", 2024-02-20T12:49:27.139 INFO:teuthology.orchestra.run.smithi071.stdout: "image_id": "46cf6318c64e842f2287ebcae09f4ed8697a6751115e5245c05af1b9b2fd053f", 2024-02-20T12:49:27.139 INFO:teuthology.orchestra.run.smithi071.stdout: "repo_digests": [ 2024-02-20T12:49:27.139 INFO:teuthology.orchestra.run.smithi071.stdout: "docker.io/ceph/daemon-base@sha256:02f4aa6b553e20e1f3fae6eefcc99f66fcf867bf0eae95d362ae2121efc26684" 2024-02-20T12:49:27.139 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-02-20T12:49:27.139 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-02-20T12:49:27.664 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-02-20T12:49:27.665 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph_version": "ceph version 16.2.5 (0883bdea7337b95e4b611c768c0279868462204a) pacific (stable)", 2024-02-20T12:49:27.665 INFO:teuthology.orchestra.run.smithi064.stdout: "image_id": "46cf6318c64e842f2287ebcae09f4ed8697a6751115e5245c05af1b9b2fd053f", 2024-02-20T12:49:27.665 INFO:teuthology.orchestra.run.smithi064.stdout: "repo_digests": [ 2024-02-20T12:49:27.665 INFO:teuthology.orchestra.run.smithi064.stdout: "docker.io/ceph/daemon-base@sha256:02f4aa6b553e20e1f3fae6eefcc99f66fcf867bf0eae95d362ae2121efc26684" 2024-02-20T12:49:27.665 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-02-20T12:49:27.665 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-02-20T12:49:27.687 DEBUG:teuthology.orchestra.run.smithi064:> sudo mkdir -p /etc/ceph 2024-02-20T12:49:27.719 DEBUG:teuthology.orchestra.run.smithi071:> sudo mkdir -p /etc/ceph 2024-02-20T12:49:27.754 DEBUG:teuthology.orchestra.run.smithi064:> sudo chmod 777 /etc/ceph 2024-02-20T12:49:27.787 DEBUG:teuthology.orchestra.run.smithi071:> sudo chmod 777 /etc/ceph 2024-02-20T12:49:27.825 INFO:tasks.cephadm:Writing seed config... 2024-02-20T12:49:27.827 INFO:tasks.cephadm: override: [osd] osd_class_default_list = * 2024-02-20T12:49:27.827 INFO:tasks.cephadm: override: [osd] osd_class_load_list = * 2024-02-20T12:49:27.827 INFO:tasks.cephadm: override: [osd] bdev async discard = True 2024-02-20T12:49:27.827 INFO:tasks.cephadm: override: [osd] bdev enable discard = True 2024-02-20T12:49:27.827 INFO:tasks.cephadm: override: [osd] bluestore allocator = bitmap 2024-02-20T12:49:27.827 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-02-20T12:49:27.827 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-02-20T12:49:27.827 INFO:tasks.cephadm: override: [osd] debug bluefs = 1/20 2024-02-20T12:49:27.827 INFO:tasks.cephadm: override: [osd] debug bluestore = 1/20 2024-02-20T12:49:27.827 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-02-20T12:49:27.827 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-02-20T12:49:27.828 INFO:tasks.cephadm: override: [osd] debug rocksdb = 4/10 2024-02-20T12:49:27.828 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-02-20T12:49:27.828 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-02-20T12:49:27.828 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-02-20T12:49:27.828 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-02-20T12:49:27.828 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-02-20T12:49:27.828 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-02-20T12:49:27.828 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-02-20T12:49:27.828 INFO:tasks.cephadm: override: [client] debug client = 20 2024-02-20T12:49:27.828 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-02-20T12:49:27.828 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-02-20T12:49:27.828 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-02-20T12:49:27.828 INFO:tasks.cephadm: override: [global] mon pg warn min per osd = 0 2024-02-20T12:49:27.828 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-02-20T12:49:27.829 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-02-20T12:49:27.829 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-02-20T12:49:27.829 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-02-20T12:49:27.829 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-02-20T12:49:27.829 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-02-20T12:49:27.829 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-02-20T12:49:27.829 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-02-20T12:49:27.829 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-02-20T12:49:27.829 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-02-20T12:49:27.829 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-02-20T12:49:27.829 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-02-20T12:49:27.829 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-02-20T12:49:27.829 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-02-20T12:49:27.829 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-02-20T12:49:27.829 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-02-20T12:49:27.830 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-02-20T12:49:27.830 DEBUG:teuthology.orchestra.run.smithi064:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-02-20T12:49:27.848 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 680ee126-cfee-11ee-95bb-87774f69a715 mon pg warn min per osd = 0 [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = true bdev debug aio = true osd sloppy crc = true osd_class_default_list = * osd_class_load_list = * bdev async discard = True bdev enable discard = True bluestore allocator = bitmap bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 1/20 debug bluestore = 1/20 debug ms = 1 debug osd = 20 debug rocksdb = 4/10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-02-20T12:49:27.849 DEBUG:teuthology.orchestra.run.smithi064:mon.smithi064> sudo journalctl -f -n 0 -u ceph-680ee126-cfee-11ee-95bb-87774f69a715@mon.smithi064.service 2024-02-20T12:49:27.891 INFO:tasks.cephadm:Bootstrapping... 2024-02-20T12:49:27.891 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/daemon-base:latest-pacific -v bootstrap --fsid 680ee126-cfee-11ee-95bb-87774f69a715 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-ip 172.21.15.64 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-02-20T12:49:27.919 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:-- Logs begin at Tue 2024-02-20 12:38:52 UTC. -- 2024-02-20T12:49:28.103 INFO:teuthology.orchestra.run.smithi064.stderr:-------------------------------------------------------------------------------- 2024-02-20T12:49:28.104 INFO:teuthology.orchestra.run.smithi064.stderr:cephadm ['--image', 'docker.io/ceph/daemon-base:latest-pacific', '-v', 'bootstrap', '--fsid', '680ee126-cfee-11ee-95bb-87774f69a715', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-ip', '172.21.15.64', '--skip-admin-label'] 2024-02-20T12:49:28.127 INFO:teuthology.orchestra.run.smithi064.stderr:/bin/podman: 4.9.3-dev 2024-02-20T12:49:28.140 INFO:teuthology.orchestra.run.smithi064.stderr:Verifying podman|docker is present... 2024-02-20T12:49:28.161 INFO:teuthology.orchestra.run.smithi064.stderr:/bin/podman: 4.9.3-dev 2024-02-20T12:49:28.170 INFO:teuthology.orchestra.run.smithi064.stderr:Verifying lvm2 is present... 2024-02-20T12:49:28.170 INFO:teuthology.orchestra.run.smithi064.stderr:Verifying time synchronization is in place... 2024-02-20T12:49:28.177 INFO:teuthology.orchestra.run.smithi064.stderr:systemctl: Failed to get unit file state for chrony.service: No such file or directory 2024-02-20T12:49:28.183 INFO:teuthology.orchestra.run.smithi064.stderr:systemctl: inactive 2024-02-20T12:49:28.189 INFO:teuthology.orchestra.run.smithi064.stderr:systemctl: enabled 2024-02-20T12:49:28.195 INFO:teuthology.orchestra.run.smithi064.stderr:systemctl: active 2024-02-20T12:49:28.195 INFO:teuthology.orchestra.run.smithi064.stderr:Unit chronyd.service is enabled and running 2024-02-20T12:49:28.195 INFO:teuthology.orchestra.run.smithi064.stderr:Repeating the final host check... 2024-02-20T12:49:28.215 INFO:teuthology.orchestra.run.smithi064.stderr:/bin/podman: 4.9.3-dev 2024-02-20T12:49:28.224 INFO:teuthology.orchestra.run.smithi064.stderr:podman (/bin/podman) version 4.9.3 is present 2024-02-20T12:49:28.224 INFO:teuthology.orchestra.run.smithi064.stderr:systemctl is present 2024-02-20T12:49:28.224 INFO:teuthology.orchestra.run.smithi064.stderr:lvcreate is present 2024-02-20T12:49:28.230 INFO:teuthology.orchestra.run.smithi064.stderr:systemctl: Failed to get unit file state for chrony.service: No such file or directory 2024-02-20T12:49:28.235 INFO:teuthology.orchestra.run.smithi064.stderr:systemctl: inactive 2024-02-20T12:49:28.241 INFO:teuthology.orchestra.run.smithi064.stderr:systemctl: enabled 2024-02-20T12:49:28.247 INFO:teuthology.orchestra.run.smithi064.stderr:systemctl: active 2024-02-20T12:49:28.248 INFO:teuthology.orchestra.run.smithi064.stderr:Unit chronyd.service is enabled and running 2024-02-20T12:49:28.248 INFO:teuthology.orchestra.run.smithi064.stderr:Host looks OK 2024-02-20T12:49:28.248 INFO:teuthology.orchestra.run.smithi064.stderr:Cluster fsid: 680ee126-cfee-11ee-95bb-87774f69a715 2024-02-20T12:49:28.248 INFO:teuthology.orchestra.run.smithi064.stderr:Acquiring lock 139689266764264 on /run/cephadm/680ee126-cfee-11ee-95bb-87774f69a715.lock 2024-02-20T12:49:28.248 INFO:teuthology.orchestra.run.smithi064.stderr:Lock 139689266764264 acquired on /run/cephadm/680ee126-cfee-11ee-95bb-87774f69a715.lock 2024-02-20T12:49:28.248 INFO:teuthology.orchestra.run.smithi064.stderr:Verifying IP 172.21.15.64 port 3300 ... 2024-02-20T12:49:28.249 INFO:teuthology.orchestra.run.smithi064.stderr:Verifying IP 172.21.15.64 port 6789 ... 2024-02-20T12:49:28.249 INFO:teuthology.orchestra.run.smithi064.stderr:Base mon IP(s) is [172.21.15.64:3300, 172.21.15.64:6789], mon addrv is [v2:172.21.15.64:3300,v1:172.21.15.64:6789] 2024-02-20T12:49:28.252 INFO:teuthology.orchestra.run.smithi064.stderr:/sbin/ip: default via 172.21.15.254 dev enp3s0f1 proto dhcp src 172.21.15.64 metric 100 2024-02-20T12:49:28.252 INFO:teuthology.orchestra.run.smithi064.stderr:/sbin/ip: 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.64 metric 100 2024-02-20T12:49:28.255 INFO:teuthology.orchestra.run.smithi064.stderr:/sbin/ip: ::1 dev lo proto kernel metric 256 pref medium 2024-02-20T12:49:28.256 INFO:teuthology.orchestra.run.smithi064.stderr:/sbin/ip: fe80::/64 dev enp3s0f1 proto kernel metric 256 pref medium 2024-02-20T12:49:28.256 INFO:teuthology.orchestra.run.smithi064.stderr:/sbin/ip: default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 1024 expires 1790sec hoplimit 64 pref medium 2024-02-20T12:49:28.258 INFO:teuthology.orchestra.run.smithi064.stderr:/sbin/ip: 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-02-20T12:49:28.258 INFO:teuthology.orchestra.run.smithi064.stderr:/sbin/ip: inet6 ::1/128 scope host 2024-02-20T12:49:28.258 INFO:teuthology.orchestra.run.smithi064.stderr:/sbin/ip: valid_lft forever preferred_lft forever 2024-02-20T12:49:28.258 INFO:teuthology.orchestra.run.smithi064.stderr:/sbin/ip: 5: enp3s0f1: mtu 1500 state UP qlen 1000 2024-02-20T12:49:28.258 INFO:teuthology.orchestra.run.smithi064.stderr:/sbin/ip: inet6 fe80::ec4:7aff:feb6:f8df/64 scope link 2024-02-20T12:49:28.258 INFO:teuthology.orchestra.run.smithi064.stderr:/sbin/ip: valid_lft forever preferred_lft forever 2024-02-20T12:49:28.259 INFO:teuthology.orchestra.run.smithi064.stderr:Mon IP `172.21.15.64` is in CIDR network `172.21.0.0/20` 2024-02-20T12:49:28.259 INFO:teuthology.orchestra.run.smithi064.stderr:Mon IP `172.21.15.64` is in CIDR network `172.21.0.0/20` 2024-02-20T12:49:28.259 INFO:teuthology.orchestra.run.smithi064.stderr:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-02-20T12:49:28.260 INFO:teuthology.orchestra.run.smithi064.stderr:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-02-20T12:49:28.261 INFO:teuthology.orchestra.run.smithi064.stderr:Pulling container image docker.io/ceph/daemon-base:latest-pacific... 2024-02-20T12:49:28.341 INFO:teuthology.orchestra.run.smithi064.stderr:/bin/podman: Trying to pull docker.io/ceph/daemon-base:latest-pacific... 2024-02-20T12:49:34.885 INFO:teuthology.orchestra.run.smithi064.stderr:/bin/podman: Getting image source signatures 2024-02-20T12:49:34.885 INFO:teuthology.orchestra.run.smithi064.stderr:/bin/podman: Copying blob sha256:a997af56d24c50f31dbc8bd557a1c2adeb9ea675d7d7142f1a37cb629070c246 2024-02-20T12:49:34.885 INFO:teuthology.orchestra.run.smithi064.stderr:/bin/podman: Copying blob sha256:7a0437f04f83f084b7ed68ad9c4a4947e12fc4e1b006b38129bac89114ec3621 2024-02-20T12:49:34.885 INFO:teuthology.orchestra.run.smithi064.stderr:/bin/podman: Copying config sha256:46cf6318c64e842f2287ebcae09f4ed8697a6751115e5245c05af1b9b2fd053f 2024-02-20T12:49:35.039 INFO:teuthology.orchestra.run.smithi064.stderr:/bin/podman: Writing manifest to image destination 2024-02-20T12:49:35.216 INFO:teuthology.orchestra.run.smithi064.stderr:/bin/podman: 46cf6318c64e842f2287ebcae09f4ed8697a6751115e5245c05af1b9b2fd053f 2024-02-20T12:49:35.843 INFO:teuthology.orchestra.run.smithi064.stderr:ceph: ceph version 16.2.5 (0883bdea7337b95e4b611c768c0279868462204a) pacific (stable) 2024-02-20T12:49:36.703 INFO:teuthology.orchestra.run.smithi064.stderr:Ceph version: ceph version 16.2.5 (0883bdea7337b95e4b611c768c0279868462204a) pacific (stable) 2024-02-20T12:49:36.703 INFO:teuthology.orchestra.run.smithi064.stderr:Extracting ceph user uid/gid from container image... 2024-02-20T12:49:37.318 INFO:teuthology.orchestra.run.smithi064.stderr:stat: 167 167 2024-02-20T12:49:38.429 INFO:teuthology.orchestra.run.smithi064.stderr:Creating initial keys... 2024-02-20T12:49:40.803 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-authtool: AQDkn9Rlf1DDKBAARDTp2JHvEG+t+b2epoGUlQ== 2024-02-20T12:49:42.051 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-authtool: AQDln9RlF+ByNxAAGYn/YXStlHKvGj9AD5QqUw== 2024-02-20T12:49:43.507 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-authtool: AQDnn9RlAMwfFxAApZbkBOit8NrgOo7r1Na1Ig== 2024-02-20T12:49:44.117 INFO:teuthology.orchestra.run.smithi064.stderr:Creating initial monmap... 2024-02-20T12:49:44.747 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/monmaptool: /usr/bin/monmaptool: monmap file /tmp/monmap 2024-02-20T12:49:44.747 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/monmaptool: /usr/bin/monmaptool: set fsid to 680ee126-cfee-11ee-95bb-87774f69a715 2024-02-20T12:49:44.747 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/monmaptool: /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-02-20T12:49:45.410 INFO:teuthology.orchestra.run.smithi064.stderr:monmaptool for smithi064 [v2:172.21.15.64:3300,v1:172.21.15.64:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-02-20T12:49:45.411 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/monmaptool: set fsid to 680ee126-cfee-11ee-95bb-87774f69a715 2024-02-20T12:49:45.411 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-02-20T12:49:45.411 INFO:teuthology.orchestra.run.smithi064.stderr: 2024-02-20T12:49:45.411 INFO:teuthology.orchestra.run.smithi064.stderr:Creating mon... 2024-02-20T12:49:46.341 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: debug 2024-02-20T12:49:46.252+0000 7efcca3fe700 0 set uid:gid to 167:167 (ceph:ceph) 2024-02-20T12:49:46.341 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: debug 2024-02-20T12:49:46.252+0000 7efcca3fe700 1 imported monmap: 2024-02-20T12:49:46.342 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: epoch 0 2024-02-20T12:49:46.342 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: fsid 680ee126-cfee-11ee-95bb-87774f69a715 2024-02-20T12:49:46.367 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: last_changed 2024-02-20T12:49:44.699059+0000 2024-02-20T12:49:46.367 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: created 2024-02-20T12:49:44.699059+0000 2024-02-20T12:49:46.367 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: min_mon_release 0 (unknown) 2024-02-20T12:49:46.368 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: election_strategy: 1 2024-02-20T12:49:46.368 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: 0: [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] mon.smithi064 2024-02-20T12:49:46.368 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: 2024-02-20T12:49:46.368 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: debug 2024-02-20T12:49:46.253+0000 7efcca3fe700 0 /usr/bin/ceph-mon: set fsid to 680ee126-cfee-11ee-95bb-87774f69a715 2024-02-20T12:49:46.368 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: debug 2024-02-20T12:49:46.253+0000 7efcca3fe700 4 rocksdb: RocksDB version: 6.8.1 2024-02-20T12:49:46.368 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: 2024-02-20T12:49:46.368 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: debug 2024-02-20T12:49:46.253+0000 7efcca3fe700 4 rocksdb: Git sha rocksdb_build_git_sha:@0@ 2024-02-20T12:49:46.368 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: debug 2024-02-20T12:49:46.253+0000 7efcca3fe700 4 rocksdb: Compile date Jul 8 2021 2024-02-20T12:49:46.368 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: debug 2024-02-20T12:49:46.253+0000 7efcca3fe700 4 rocksdb: DB SUMMARY 2024-02-20T12:49:46.368 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: 2024-02-20T12:49:46.368 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: debug 2024-02-20T12:49:46.253+0000 7efcca3fe700 4 rocksdb: SST files in /var/lib/ceph/mon/ceph-smithi064/store.db dir, Total Num: 0, files: 2024-02-20T12:49:46.368 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: 2024-02-20T12:49:46.368 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: debug 2024-02-20T12:49:46.253+0000 7efcca3fe700 4 rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-smithi064/store.db: 2024-02-20T12:49:46.368 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: 2024-02-20T12:49:46.369 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: debug 2024-02-20T12:49:46.253+0000 7efcca3fe700 4 rocksdb: Options.error_if_exists: 0 2024-02-20T12:49:46.369 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: debug 2024-02-20T12:49:46.253+0000 7efcca3fe700 4 rocksdb: Options.create_if_missing: 1 2024-02-20T12:49:46.369 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: debug 2024-02-20T12:49:46.253+0000 7efcca3fe700 4 rocksdb: Options.paranoid_checks: 1 2024-02-20T12:49:46.369 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: debug 2024-02-20T12:49:46.253+0000 7efcca3fe700 4 rocksdb: Options.env: 0x559c7eaf01c0 2024-02-20T12:49:46.369 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: debug 2024-02-20T12:49:46.253+0000 7efcca3fe700 4 rocksdb: Options.fs: Posix File System 2024-02-20T12:49:46.369 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: debug 2024-02-20T12:49:46.253+0000 7efcca3fe700 4 rocksdb: Options.info_log: 0x559c8076cc20 2024-02-20T12:49:46.369 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: debug 2024-02-20T12:49:46.253+0000 7efcca3fe700 4 rocksdb: Options.max_file_opening_threads: 16 2024-02-20T12:49:46.369 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: debug 2024-02-20T12:49:46.253+0000 7efcca3fe700 4 rocksdb: Options.statistics: (nil) 2024-02-20T12:49:46.369 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: debug 2024-02-20T12:49:46.253+0000 7efcca3fe700 4 rocksdb: Options.use_fsync: 0 2024-02-20T12:49:46.369 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: debug 2024-02-20T12:49:46.253+0000 7efcca3fe700 4 rocksdb: Options.max_log_file_size: 0 2024-02-20T12:49:46.369 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: debug 2024-02-20T12:49:46.253+0000 7efcca3fe700 4 rocksdb: Options.max_manifest_file_size: 1073741824 2024-02-20T12:49:46.369 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: debug 2024-02-20T12:49:46.253+0000 7efcca3fe700 4 rocksdb: Options.log_file_time_to_roll: 0 2024-02-20T12:49:46.369 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: debug 2024-02-20T12:49:46.253+0000 7efcca3fe700 4 rocksdb: Options.keep_log_file_num: 1000 2024-02-20T12:49:46.369 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: debug 2024-02-20T12:49:46.253+0000 7efcca3fe700 4 rocksdb: Options.recycle_log_file_num: 0 2024-02-20T12:49:46.369 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: debug 2024-02-20T12:49:46.253+0000 7efcca3fe700 4 rocksdb: Options.allow_fallocate: 1 2024-02-20T12:49:46.369 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: debug 2024-02-20T12:49:46.253+0000 7efcca3fe700 4 rocksdb: Options.allow_mmap_reads: 0 2024-02-20T12:49:46.369 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: debug 2024-02-20T12:49:46.253+0000 7efcca3fe700 4 rocksdb: Options.allow_mmap_writes: 0 2024-02-20T12:49:46.370 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: debug 2024-02-20T12:49:46.253+0000 7efcca3fe700 4 rocksdb: Options.use_direct_reads: 0 2024-02-20T12:49:46.370 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: debug 2024-02-20T12:49:46.253+0000 7efcca3fe700 4 rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-02-20T12:49:46.370 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: debug 2024-02-20T12:49:46.254+0000 7efcca3fe700 4 rocksdb: Options.create_missing_column_families: 0 2024-02-20T12:49:46.370 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: debug 2024-02-20T12:49:46.254+0000 7efcca3fe700 4 rocksdb: Options.db_log_dir: 2024-02-20T12:49:46.370 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: debug 2024-02-20T12:49:46.254+0000 7efcca3fe700 4 rocksdb: Options.wal_dir: /var/lib/ceph/mon/ceph-smithi064/store.db 2024-02-20T12:49:46.370 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: debug 2024-02-20T12:49:46.254+0000 7efcca3fe700 4 rocksdb: Options.table_cache_numshardbits: 6 2024-02-20T12:49:46.370 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: debug 2024-02-20T12:49:46.254+0000 7efcca3fe700 4 rocksdb: Options.max_subcompactions: 1 2024-02-20T12:49:46.370 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: debug 2024-02-20T12:49:46.254+0000 7efcca3fe700 4 rocksdb: Options.max_background_flushes: -1 2024-02-20T12:49:46.370 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: debug 2024-02-20T12:49:46.254+0000 7efcca3fe700 4 rocksdb: Options.WAL_ttl_seconds: 0 2024-02-20T12:49:46.370 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: debug 2024-02-20T12:49:46.254+0000 7efcca3fe700 4 rocksdb: Options.WAL_size_limit_MB: 0 2024-02-20T12:49:46.370 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: debug 2024-02-20T12:49:46.254+0000 7efcca3fe700 4 rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-02-20T12:49:46.370 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: debug 2024-02-20T12:49:46.254+0000 7efcca3fe700 4 rocksdb: Options.manifest_preallocation_size: 4194304 2024-02-20T12:49:46.370 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: debug 2024-02-20T12:49:46.254+0000 7efcca3fe700 4 rocksdb: Options.is_fd_close_on_exec: 1 2024-02-20T12:49:46.370 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: debug 2024-02-20T12:49:46.254+0000 7efcca3fe700 4 rocksdb: Options.advise_random_on_open: 1 2024-02-20T12:49:46.370 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: debug 2024-02-20T12:49:46.254+0000 7efcca3fe700 4 rocksdb: Options.db_write_buffer_size: 0 2024-02-20T12:49:46.371 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: debug 2024-02-20T12:49:46.254+0000 7efcca3fe700 4 rocksdb: Options.write_buffer_manager: 0x559c80775530 2024-02-20T12:49:46.371 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: debug 2024-02-20T12:49:46.254+0000 7efcca3fe700 4 rocksdb: Options.access_hint_on_compaction_start: 1 2024-02-20T12:49:46.371 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: debug 2024-02-20T12:49:46.254+0000 7efcca3fe700 4 rocksdb: Options.new_table_reader_for_compaction_inputs: 0 2024-02-20T12:49:46.371 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: debug 2024-02-20T12:49:46.254+0000 7efcca3fe700 4 rocksdb: Options.random_access_max_buffer_size: 1048576 2024-02-20T12:49:46.371 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: debug 2024-02-20T12:49:46.254+0000 7efcca3fe700 4 rocksdb: Options.use_adaptive_mutex: 0 2024-02-20T12:49:46.371 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: debug 2024-02-20T12:49:46.254+0000 7efcca3fe700 4 rocksdb: Options.rate_limiter: (nil) 2024-02-20T12:49:46.371 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: debug 2024-02-20T12:49:46.254+0000 7efcca3fe700 4 rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-02-20T12:49:46.371 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: debug 2024-02-20T12:49:46.254+0000 7efcca3fe700 4 rocksdb: Options.wal_recovery_mode: 2 2024-02-20T12:49:46.371 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: debug 2024-02-20T12:49:46.254+0000 7efcca3fe700 4 rocksdb: Options.enable_thread_tracking: 0 2024-02-20T12:49:46.371 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: debug 2024-02-20T12:49:46.254+0000 7efcca3fe700 4 rocksdb: Options.enable_pipelined_write: 0 2024-02-20T12:49:46.371 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: debug 2024-02-20T12:49:46.254+0000 7efcca3fe700 4 rocksdb: Options.unordered_write: 0 2024-02-20T12:49:46.371 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: debug 2024-02-20T12:49:46.254+0000 7efcca3fe700 4 rocksdb: Options.allow_concurrent_memtable_write: 1 2024-02-20T12:49:46.371 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: debug 2024-02-20T12:49:46.254+0000 7efcca3fe700 4 rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-02-20T12:49:46.371 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: debug 2024-02-20T12:49:46.254+0000 7efcca3fe700 4 rocksdb: Options.write_thread_max_yield_usec: 100 2024-02-20T12:49:46.371 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: debug 2024-02-20T12:49:46.254+0000 7efcca3fe700 4 rocksdb: Options.write_thread_slow_yield_usec: 3 2024-02-20T12:49:46.371 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: debug 2024-02-20T12:49:46.254+0000 7efcca3fe700 4 rocksdb: Options.row_cache: None 2024-02-20T12:49:46.372 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: debug 2024-02-20T12:49:46.254+0000 7efcca3fe700 4 rocksdb: Options.wal_filter: None 2024-02-20T12:49:46.372 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: debug 2024-02-20T12:49:46.254+0000 7efcca3fe700 4 rocksdb: Options.avoid_flush_during_recovery: 0 2024-02-20T12:49:46.372 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: debug 2024-02-20T12:49:46.254+0000 7efcca3fe700 4 rocksdb: Options.allow_ingest_behind: 0 2024-02-20T12:49:46.372 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: debug 2024-02-20T12:49:46.254+0000 7efcca3fe700 4 rocksdb: Options.preserve_deletes: 0 2024-02-20T12:49:46.372 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: debug 2024-02-20T12:49:46.254+0000 7efcca3fe700 4 rocksdb: Options.two_write_queues: 0 2024-02-20T12:49:46.372 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: debug 2024-02-20T12:49:46.254+0000 7efcca3fe700 4 rocksdb: Options.manual_wal_flush: 0 2024-02-20T12:49:46.372 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: debug 2024-02-20T12:49:46.254+0000 7efcca3fe700 4 rocksdb: Options.atomic_flush: 0 2024-02-20T12:49:46.372 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: debug 2024-02-20T12:49:46.254+0000 7efcca3fe700 4 rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-02-20T12:49:46.372 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: debug 2024-02-20T12:49:46.254+0000 7efcca3fe700 4 rocksdb: Options.persist_stats_to_disk: 0 2024-02-20T12:49:46.372 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: debug 2024-02-20T12:49:46.254+0000 7efcca3fe700 4 rocksdb: Options.write_dbid_to_manifest: 0 2024-02-20T12:49:46.372 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: debug 2024-02-20T12:49:46.254+0000 7efcca3fe700 4 rocksdb: Options.log_readahead_size: 0 2024-02-20T12:49:46.372 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: debug 2024-02-20T12:49:46.254+0000 7efcca3fe700 4 rocksdb: Options.sst_file_checksum_func: Unknown 2024-02-20T12:49:46.372 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: debug 2024-02-20T12:49:46.254+0000 7efcca3fe700 4 rocksdb: Options.max_background_jobs: 2 2024-02-20T12:49:46.372 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: debug 2024-02-20T12:49:46.254+0000 7efcca3fe700 4 rocksdb: Options.max_background_compactions: -1 2024-02-20T12:49:46.373 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: debug 2024-02-20T12:49:46.254+0000 7efcca3fe700 4 rocksdb: Options.avoid_flush_during_shutdown: 0 2024-02-20T12:49:46.373 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: debug 2024-02-20T12:49:46.254+0000 7efcca3fe700 4 rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-02-20T12:49:46.373 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: debug 2024-02-20T12:49:46.254+0000 7efcca3fe700 4 rocksdb: Options.delayed_write_rate : 16777216 2024-02-20T12:49:46.373 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: debug 2024-02-20T12:49:46.254+0000 7efcca3fe700 4 rocksdb: Options.max_total_wal_size: 0 2024-02-20T12:49:46.373 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: debug 2024-02-20T12:49:46.254+0000 7efcca3fe700 4 rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-02-20T12:49:46.373 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: debug 2024-02-20T12:49:46.254+0000 7efcca3fe700 4 rocksdb: Options.stats_dump_period_sec: 600 2024-02-20T12:49:46.373 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: debug 2024-02-20T12:49:46.254+0000 7efcca3fe700 4 rocksdb: Options.stats_persist_period_sec: 600 2024-02-20T12:49:46.373 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: debug 2024-02-20T12:49:46.254+0000 7efcca3fe700 4 rocksdb: Options.stats_history_buffer_size: 1048576 2024-02-20T12:49:46.373 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: debug 2024-02-20T12:49:46.254+0000 7efcca3fe700 4 rocksdb: Options.max_open_files: -1 2024-02-20T12:49:46.373 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: debug 2024-02-20T12:49:46.254+0000 7efcca3fe700 4 rocksdb: Options.bytes_per_sync: 0 2024-02-20T12:49:46.373 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: debug 2024-02-20T12:49:46.254+0000 7efcca3fe700 4 rocksdb: Options.wal_bytes_per_sync: 0 2024-02-20T12:49:46.373 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: debug 2024-02-20T12:49:46.254+0000 7efcca3fe700 4 rocksdb: Options.strict_bytes_per_sync: 0 2024-02-20T12:49:46.373 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: debug 2024-02-20T12:49:46.254+0000 7efcca3fe700 4 rocksdb: Options.compaction_readahead_size: 0 2024-02-20T12:49:46.373 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: debug 2024-02-20T12:49:46.254+0000 7efcca3fe700 4 rocksdb: Compression algorithms supported: 2024-02-20T12:49:46.374 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: debug 2024-02-20T12:49:46.254+0000 7efcca3fe700 4 rocksdb: kZSTDNotFinalCompression supported: 0 2024-02-20T12:49:46.374 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: debug 2024-02-20T12:49:46.254+0000 7efcca3fe700 4 rocksdb: kZSTD supported: 0 2024-02-20T12:49:46.374 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: debug 2024-02-20T12:49:46.254+0000 7efcca3fe700 4 rocksdb: kXpressCompression supported: 0 2024-02-20T12:49:46.374 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: debug 2024-02-20T12:49:46.254+0000 7efcca3fe700 4 rocksdb: kLZ4HCCompression supported: 1 2024-02-20T12:49:46.374 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: debug 2024-02-20T12:49:46.254+0000 7efcca3fe700 4 rocksdb: kLZ4Compression supported: 1 2024-02-20T12:49:46.374 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: debug 2024-02-20T12:49:46.254+0000 7efcca3fe700 4 rocksdb: kBZip2Compression supported: 0 2024-02-20T12:49:46.374 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: debug 2024-02-20T12:49:46.254+0000 7efcca3fe700 4 rocksdb: kZlibCompression supported: 1 2024-02-20T12:49:46.374 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: debug 2024-02-20T12:49:46.254+0000 7efcca3fe700 4 rocksdb: kSnappyCompression supported: 1 2024-02-20T12:49:46.374 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: debug 2024-02-20T12:49:46.254+0000 7efcca3fe700 4 rocksdb: Fast CRC32 supported: Supported on x86 2024-02-20T12:49:46.374 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: debug 2024-02-20T12:49:46.254+0000 7efcca3fe700 4 rocksdb: [db_impl/db_impl_open.cc:273] Creating manifest 1 2024-02-20T12:49:46.374 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: 2024-02-20T12:49:46.374 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: debug 2024-02-20T12:49:46.254+0000 7efcca3fe700 4 rocksdb: [version_set.cc:4413] Recovering from manifest file: /var/lib/ceph/mon/ceph-smithi064/store.db/MANIFEST-000001 2024-02-20T12:49:46.374 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: 2024-02-20T12:49:46.374 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: debug 2024-02-20T12:49:46.254+0000 7efcca3fe700 4 rocksdb: [column_family.cc:552] --------------- Options for column family [default]: 2024-02-20T12:49:46.374 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: 2024-02-20T12:49:46.375 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: debug 2024-02-20T12:49:46.255+0000 7efcca3fe700 4 rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-02-20T12:49:46.375 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: debug 2024-02-20T12:49:46.255+0000 7efcca3fe700 4 rocksdb: Options.merge_operator: 2024-02-20T12:49:46.375 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: debug 2024-02-20T12:49:46.255+0000 7efcca3fe700 4 rocksdb: Options.compaction_filter: None 2024-02-20T12:49:46.375 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: debug 2024-02-20T12:49:46.255+0000 7efcca3fe700 4 rocksdb: Options.compaction_filter_factory: None 2024-02-20T12:49:46.375 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: debug 2024-02-20T12:49:46.255+0000 7efcca3fe700 4 rocksdb: Options.memtable_factory: SkipListFactory 2024-02-20T12:49:46.375 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: debug 2024-02-20T12:49:46.255+0000 7efcca3fe700 4 rocksdb: Options.table_factory: BlockBasedTable 2024-02-20T12:49:46.375 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: debug 2024-02-20T12:49:46.255+0000 7efcca3fe700 4 rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x559c80686160) 2024-02-20T12:49:46.375 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: cache_index_and_filter_blocks: 1 2024-02-20T12:49:46.375 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: cache_index_and_filter_blocks_with_high_priority: 0 2024-02-20T12:49:46.375 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: pin_l0_filter_and_index_blocks_in_cache: 0 2024-02-20T12:49:46.375 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: pin_top_level_index_and_filter: 1 2024-02-20T12:49:46.375 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: index_type: 0 2024-02-20T12:49:46.375 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: data_block_index_type: 0 2024-02-20T12:49:46.375 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: index_shortening: 1 2024-02-20T12:49:46.375 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: data_block_hash_table_util_ratio: 0.750000 2024-02-20T12:49:46.376 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: hash_index_allow_collision: 1 2024-02-20T12:49:46.376 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: checksum: 1 2024-02-20T12:49:46.376 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: no_block_cache: 0 2024-02-20T12:49:46.376 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: block_cache: 0x559c806bcd10 2024-02-20T12:49:46.376 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: block_cache_name: BinnedLRUCache 2024-02-20T12:49:46.376 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: block_cache_options: 2024-02-20T12:49:46.376 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: capacity : 536870912 2024-02-20T12:49:46.376 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: num_shard_bits : 4 2024-02-20T12:49:46.376 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: strict_capacity_limit : 0 2024-02-20T12:49:46.376 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: high_pri_pool_ratio: 0.000 2024-02-20T12:49:46.376 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: block_cache_compressed: (nil) 2024-02-20T12:49:46.376 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: persistent_cache: (nil) 2024-02-20T12:49:46.376 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: block_size: 4096 2024-02-20T12:49:46.376 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: block_size_deviation: 10 2024-02-20T12:49:46.377 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: block_restart_interval: 16 2024-02-20T12:49:46.377 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: index_block_restart_interval: 1 2024-02-20T12:49:46.377 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: metadata_block_size: 4096 2024-02-20T12:49:46.377 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: partition_filters: 0 2024-02-20T12:49:46.377 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: use_delta_encoding: 1 2024-02-20T12:49:46.377 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: filter_policy: rocksdb.BuiltinBloomFilter 2024-02-20T12:49:46.377 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: whole_key_filtering: 1 2024-02-20T12:49:46.377 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: verify_compression: 0 2024-02-20T12:49:46.377 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: read_amp_bytes_per_bit: 0 2024-02-20T12:49:46.377 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: format_version: 2 2024-02-20T12:49:46.377 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: enable_index_compression: 1 2024-02-20T12:49:46.377 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: block_align: 0 2024-02-20T12:49:46.377 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: 2024-02-20T12:49:46.377 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: debug 2024-02-20T12:49:46.255+0000 7efcca3fe700 4 rocksdb: Options.write_buffer_size: 33554432 2024-02-20T12:49:46.378 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: debug 2024-02-20T12:49:46.255+0000 7efcca3fe700 4 rocksdb: Options.max_write_buffer_number: 2 2024-02-20T12:49:46.378 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: debug 2024-02-20T12:49:46.255+0000 7efcca3fe700 4 rocksdb: Options.compression: NoCompression 2024-02-20T12:49:46.378 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: debug 2024-02-20T12:49:46.255+0000 7efcca3fe700 4 rocksdb: Options.bottommost_compression: Disabled 2024-02-20T12:49:46.378 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: debug 2024-02-20T12:49:46.255+0000 7efcca3fe700 4 rocksdb: Options.prefix_extractor: nullptr 2024-02-20T12:49:46.378 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: debug 2024-02-20T12:49:46.255+0000 7efcca3fe700 4 rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-02-20T12:49:46.378 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: debug 2024-02-20T12:49:46.255+0000 7efcca3fe700 4 rocksdb: Options.num_levels: 7 2024-02-20T12:49:46.378 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: debug 2024-02-20T12:49:46.255+0000 7efcca3fe700 4 rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-02-20T12:49:46.378 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: debug 2024-02-20T12:49:46.255+0000 7efcca3fe700 4 rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-02-20T12:49:46.378 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: debug 2024-02-20T12:49:46.255+0000 7efcca3fe700 4 rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-02-20T12:49:46.378 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: debug 2024-02-20T12:49:46.255+0000 7efcca3fe700 4 rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-02-20T12:49:46.378 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: debug 2024-02-20T12:49:46.255+0000 7efcca3fe700 4 rocksdb: Options.bottommost_compression_opts.level: 32767 2024-02-20T12:49:46.378 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: debug 2024-02-20T12:49:46.255+0000 7efcca3fe700 4 rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-02-20T12:49:46.378 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: debug 2024-02-20T12:49:46.255+0000 7efcca3fe700 4 rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-02-20T12:49:46.379 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: debug 2024-02-20T12:49:46.255+0000 7efcca3fe700 4 rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-02-20T12:49:46.379 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: debug 2024-02-20T12:49:46.255+0000 7efcca3fe700 4 rocksdb: Options.bottommost_compression_opts.enabled: false 2024-02-20T12:49:46.379 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: debug 2024-02-20T12:49:46.255+0000 7efcca3fe700 4 rocksdb: Options.compression_opts.window_bits: -14 2024-02-20T12:49:46.379 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: debug 2024-02-20T12:49:46.255+0000 7efcca3fe700 4 rocksdb: Options.compression_opts.level: 32767 2024-02-20T12:49:46.379 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: debug 2024-02-20T12:49:46.255+0000 7efcca3fe700 4 rocksdb: Options.compression_opts.strategy: 0 2024-02-20T12:49:46.379 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: debug 2024-02-20T12:49:46.255+0000 7efcca3fe700 4 rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-02-20T12:49:46.379 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: debug 2024-02-20T12:49:46.255+0000 7efcca3fe700 4 rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-02-20T12:49:46.379 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: debug 2024-02-20T12:49:46.255+0000 7efcca3fe700 4 rocksdb: Options.compression_opts.enabled: false 2024-02-20T12:49:46.379 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: debug 2024-02-20T12:49:46.255+0000 7efcca3fe700 4 rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-02-20T12:49:46.379 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: debug 2024-02-20T12:49:46.255+0000 7efcca3fe700 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-02-20T12:49:46.379 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: debug 2024-02-20T12:49:46.255+0000 7efcca3fe700 4 rocksdb: Options.level0_stop_writes_trigger: 36 2024-02-20T12:49:46.379 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: debug 2024-02-20T12:49:46.255+0000 7efcca3fe700 4 rocksdb: Options.target_file_size_base: 67108864 2024-02-20T12:49:46.380 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: debug 2024-02-20T12:49:46.255+0000 7efcca3fe700 4 rocksdb: Options.target_file_size_multiplier: 1 2024-02-20T12:49:46.380 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: debug 2024-02-20T12:49:46.255+0000 7efcca3fe700 4 rocksdb: Options.max_bytes_for_level_base: 268435456 2024-02-20T12:49:46.380 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: debug 2024-02-20T12:49:46.255+0000 7efcca3fe700 4 rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-02-20T12:49:46.380 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: debug 2024-02-20T12:49:46.255+0000 7efcca3fe700 4 rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-02-20T12:49:46.380 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: debug 2024-02-20T12:49:46.255+0000 7efcca3fe700 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-02-20T12:49:46.380 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: debug 2024-02-20T12:49:46.255+0000 7efcca3fe700 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-02-20T12:49:46.380 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: debug 2024-02-20T12:49:46.255+0000 7efcca3fe700 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-02-20T12:49:46.380 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: debug 2024-02-20T12:49:46.255+0000 7efcca3fe700 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-02-20T12:49:46.380 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: debug 2024-02-20T12:49:46.255+0000 7efcca3fe700 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-02-20T12:49:46.380 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: debug 2024-02-20T12:49:46.255+0000 7efcca3fe700 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-02-20T12:49:46.380 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: debug 2024-02-20T12:49:46.255+0000 7efcca3fe700 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-02-20T12:49:46.380 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: debug 2024-02-20T12:49:46.255+0000 7efcca3fe700 4 rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-02-20T12:49:46.380 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: debug 2024-02-20T12:49:46.255+0000 7efcca3fe700 4 rocksdb: Options.max_compaction_bytes: 1677721600 2024-02-20T12:49:46.381 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: debug 2024-02-20T12:49:46.255+0000 7efcca3fe700 4 rocksdb: Options.arena_block_size: 4194304 2024-02-20T12:49:46.381 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: debug 2024-02-20T12:49:46.255+0000 7efcca3fe700 4 rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-02-20T12:49:46.381 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: debug 2024-02-20T12:49:46.255+0000 7efcca3fe700 4 rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-02-20T12:49:46.381 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: debug 2024-02-20T12:49:46.255+0000 7efcca3fe700 4 rocksdb: Options.rate_limit_delay_max_milliseconds: 100 2024-02-20T12:49:46.381 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: debug 2024-02-20T12:49:46.255+0000 7efcca3fe700 4 rocksdb: Options.disable_auto_compactions: 0 2024-02-20T12:49:46.381 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: debug 2024-02-20T12:49:46.255+0000 7efcca3fe700 4 rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-02-20T12:49:46.381 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: debug 2024-02-20T12:49:46.255+0000 7efcca3fe700 4 rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-02-20T12:49:46.381 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: debug 2024-02-20T12:49:46.255+0000 7efcca3fe700 4 rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-02-20T12:49:46.381 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: debug 2024-02-20T12:49:46.255+0000 7efcca3fe700 4 rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-02-20T12:49:46.381 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: debug 2024-02-20T12:49:46.255+0000 7efcca3fe700 4 rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-02-20T12:49:46.381 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: debug 2024-02-20T12:49:46.255+0000 7efcca3fe700 4 rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-02-20T12:49:46.381 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: debug 2024-02-20T12:49:46.255+0000 7efcca3fe700 4 rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-02-20T12:49:46.381 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: debug 2024-02-20T12:49:46.255+0000 7efcca3fe700 4 rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-02-20T12:49:46.381 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: debug 2024-02-20T12:49:46.255+0000 7efcca3fe700 4 rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-02-20T12:49:46.382 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: debug 2024-02-20T12:49:46.255+0000 7efcca3fe700 4 rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-02-20T12:49:46.382 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: debug 2024-02-20T12:49:46.255+0000 7efcca3fe700 4 rocksdb: Options.table_properties_collectors: 2024-02-20T12:49:46.382 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: debug 2024-02-20T12:49:46.255+0000 7efcca3fe700 4 rocksdb: Options.inplace_update_support: 0 2024-02-20T12:49:46.382 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: debug 2024-02-20T12:49:46.255+0000 7efcca3fe700 4 rocksdb: Options.inplace_update_num_locks: 10000 2024-02-20T12:49:46.382 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: debug 2024-02-20T12:49:46.255+0000 7efcca3fe700 4 rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-02-20T12:49:46.382 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: debug 2024-02-20T12:49:46.255+0000 7efcca3fe700 4 rocksdb: Options.memtable_whole_key_filtering: 0 2024-02-20T12:49:46.382 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: debug 2024-02-20T12:49:46.255+0000 7efcca3fe700 4 rocksdb: Options.memtable_huge_page_size: 0 2024-02-20T12:49:46.382 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: debug 2024-02-20T12:49:46.255+0000 7efcca3fe700 4 rocksdb: Options.bloom_locality: 0 2024-02-20T12:49:46.382 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: debug 2024-02-20T12:49:46.255+0000 7efcca3fe700 4 rocksdb: Options.max_successive_merges: 0 2024-02-20T12:49:46.382 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: debug 2024-02-20T12:49:46.255+0000 7efcca3fe700 4 rocksdb: Options.optimize_filters_for_hits: 0 2024-02-20T12:49:46.382 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: debug 2024-02-20T12:49:46.255+0000 7efcca3fe700 4 rocksdb: Options.paranoid_file_checks: 0 2024-02-20T12:49:46.382 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: debug 2024-02-20T12:49:46.255+0000 7efcca3fe700 4 rocksdb: Options.force_consistency_checks: 0 2024-02-20T12:49:46.382 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: debug 2024-02-20T12:49:46.255+0000 7efcca3fe700 4 rocksdb: Options.report_bg_io_stats: 0 2024-02-20T12:49:46.383 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: debug 2024-02-20T12:49:46.255+0000 7efcca3fe700 4 rocksdb: Options.ttl: 2592000 2024-02-20T12:49:46.383 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: debug 2024-02-20T12:49:46.255+0000 7efcca3fe700 4 rocksdb: Options.periodic_compaction_seconds: 0 2024-02-20T12:49:46.383 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: debug 2024-02-20T12:49:46.255+0000 7efcca3fe700 4 rocksdb: [version_set.cc:4568] Recovered from manifest file:/var/lib/ceph/mon/ceph-smithi064/store.db/MANIFEST-000001 succeeded,manifest_file_number is 1, next_file_number is 3, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-02-20T12:49:46.383 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: 2024-02-20T12:49:46.383 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: debug 2024-02-20T12:49:46.255+0000 7efcca3fe700 4 rocksdb: [version_set.cc:4577] Column family [default] (ID 0), log number is 0 2024-02-20T12:49:46.383 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: 2024-02-20T12:49:46.383 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: debug 2024-02-20T12:49:46.256+0000 7efcca3fe700 4 rocksdb: DB pointer 0x559c80781800 2024-02-20T12:49:46.383 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: debug 2024-02-20T12:49:46.256+0000 7efcb3287700 4 rocksdb: [db_impl/db_impl.cc:850] ------- DUMPING STATS ------- 2024-02-20T12:49:46.383 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: debug 2024-02-20T12:49:46.256+0000 7efcb3287700 4 rocksdb: [db_impl/db_impl.cc:851] 2024-02-20T12:49:46.383 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: ** DB Stats ** 2024-02-20T12:49:46.383 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: Uptime(secs): 0.0 total, 0.0 interval 2024-02-20T12:49:46.383 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-02-20T12:49:46.383 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-02-20T12:49:46.383 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-02-20T12:49:46.384 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-02-20T12:49:46.384 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 MB, 0.00 MB/s 2024-02-20T12:49:46.384 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-02-20T12:49:46.384 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: 2024-02-20T12:49:46.384 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: ** Compaction Stats [default] ** 2024-02-20T12:49:46.384 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-02-20T12:49:46.384 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-02-20T12:49:46.384 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: Sum 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 2024-02-20T12:49:46.384 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 2024-02-20T12:49:46.384 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: 2024-02-20T12:49:46.384 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: ** Compaction Stats [default] ** 2024-02-20T12:49:46.384 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-02-20T12:49:46.384 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-02-20T12:49:46.384 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: Uptime(secs): 0.0 total, 0.0 interval 2024-02-20T12:49:46.385 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: Flush(GB): cumulative 0.000, interval 0.000 2024-02-20T12:49:46.385 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: AddFile(GB): cumulative 0.000, interval 0.000 2024-02-20T12:49:46.385 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: AddFile(Total Files): cumulative 0, interval 0 2024-02-20T12:49:46.385 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: AddFile(L0 Files): cumulative 0, interval 0 2024-02-20T12:49:46.385 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: AddFile(Keys): cumulative 0, interval 0 2024-02-20T12:49:46.385 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: Cumulative compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-20T12:49:46.385 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: Interval compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-20T12:49:46.385 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-02-20T12:49:46.385 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: 2024-02-20T12:49:46.385 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: ** File Read Latency Histogram By Level [default] ** 2024-02-20T12:49:46.385 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: 2024-02-20T12:49:46.385 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: ** Compaction Stats [default] ** 2024-02-20T12:49:46.385 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-02-20T12:49:46.385 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-02-20T12:49:46.385 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: Sum 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 2024-02-20T12:49:46.386 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 2024-02-20T12:49:46.386 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: 2024-02-20T12:49:46.386 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: ** Compaction Stats [default] ** 2024-02-20T12:49:46.386 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-02-20T12:49:46.386 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-02-20T12:49:46.386 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: Uptime(secs): 0.0 total, 0.0 interval 2024-02-20T12:49:46.386 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: Flush(GB): cumulative 0.000, interval 0.000 2024-02-20T12:49:46.386 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: AddFile(GB): cumulative 0.000, interval 0.000 2024-02-20T12:49:46.386 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: AddFile(Total Files): cumulative 0, interval 0 2024-02-20T12:49:46.386 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: AddFile(L0 Files): cumulative 0, interval 0 2024-02-20T12:49:46.386 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: AddFile(Keys): cumulative 0, interval 0 2024-02-20T12:49:46.386 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: Cumulative compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-20T12:49:46.386 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: Interval compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-20T12:49:46.386 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-02-20T12:49:46.386 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: 2024-02-20T12:49:46.387 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: ** File Read Latency Histogram By Level [default] ** 2024-02-20T12:49:46.387 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: 2024-02-20T12:49:46.387 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: debug 2024-02-20T12:49:46.258+0000 7efcca3fe700 4 rocksdb: [db_impl/db_impl.cc:397] Shutdown: canceling all background work 2024-02-20T12:49:46.387 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: debug 2024-02-20T12:49:46.258+0000 7efcca3fe700 4 rocksdb: [db_impl/db_impl.cc:573] Shutdown complete 2024-02-20T12:49:46.387 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph-mon: debug 2024-02-20T12:49:46.258+0000 7efcca3fe700 0 /usr/bin/ceph-mon: created monfs at /var/lib/ceph/mon/ceph-smithi064 for mon.smithi064 2024-02-20T12:49:47.011 INFO:teuthology.orchestra.run.smithi064.stderr:create mon.smithi064 on 2024-02-20T12:49:47.026 INFO:teuthology.orchestra.run.smithi064.stderr:systemctl: Removed /etc/systemd/system/multi-user.target.wants/ceph.target. 2024-02-20T12:49:47.128 INFO:teuthology.orchestra.run.smithi064.stderr:systemctl: Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-02-20T12:49:47.230 INFO:teuthology.orchestra.run.smithi064.stderr:systemctl: Created symlink /etc/systemd/system/multi-user.target.wants/ceph-680ee126-cfee-11ee-95bb-87774f69a715.target → /etc/systemd/system/ceph-680ee126-cfee-11ee-95bb-87774f69a715.target. 2024-02-20T12:49:47.230 INFO:teuthology.orchestra.run.smithi064.stderr:systemctl: Created symlink /etc/systemd/system/ceph.target.wants/ceph-680ee126-cfee-11ee-95bb-87774f69a715.target → /etc/systemd/system/ceph-680ee126-cfee-11ee-95bb-87774f69a715.target. 2024-02-20T12:49:47.451 INFO:teuthology.orchestra.run.smithi064.stderr:systemctl: Failed to reset failed state of unit ceph-680ee126-cfee-11ee-95bb-87774f69a715@mon.smithi064.service: Unit ceph-680ee126-cfee-11ee-95bb-87774f69a715@mon.smithi064.service not loaded. 2024-02-20T12:49:47.458 INFO:teuthology.orchestra.run.smithi064.stderr:systemctl: Created symlink /etc/systemd/system/ceph-680ee126-cfee-11ee-95bb-87774f69a715.target.wants/ceph-680ee126-cfee-11ee-95bb-87774f69a715@mon.smithi064.service → /etc/systemd/system/ceph-680ee126-cfee-11ee-95bb-87774f69a715@.service. 2024-02-20T12:49:47.586 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:49:47 smithi064 systemd[1]: Starting Ceph mon.smithi064 for 680ee126-cfee-11ee-95bb-87774f69a715... 2024-02-20T12:49:48.223 INFO:teuthology.orchestra.run.smithi064.stderr:systemctl: disabled 2024-02-20T12:49:48.231 INFO:teuthology.orchestra.run.smithi064.stderr:systemctl: inactive 2024-02-20T12:49:48.232 INFO:teuthology.orchestra.run.smithi064.stderr:firewalld.service is not enabled 2024-02-20T12:49:48.232 INFO:teuthology.orchestra.run.smithi064.stderr:Not possible to enable service . firewalld.service is not available 2024-02-20T12:49:48.232 INFO:teuthology.orchestra.run.smithi064.stderr:Waiting for mon to start... 2024-02-20T12:49:48.232 INFO:teuthology.orchestra.run.smithi064.stderr:Waiting for mon... 2024-02-20T12:49:48.455 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:49:48 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[37847]: cluster 2024-02-20T12:49:48.180727+0000 mon.smithi064 (mon.0) 0 : [INF] mkfs 680ee126-cfee-11ee-95bb-87774f69a715 2024-02-20T12:49:48.456 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:49:48 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[37847]: cluster 2024-02-20T12:49:48.163769+0000 mon.smithi064 (mon.0) 1 : cluster [INF] mon.smithi064 is new leader, mons smithi064 in quorum (ranks 0) 2024-02-20T12:49:48.456 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:49:48 smithi064 bash[37831]: e9370c6141852c30e3a34ac24121455107b29e274f19d9f08f681639aea53188 2024-02-20T12:49:48.456 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:49:48 smithi064 systemd[1]: Started Ceph mon.smithi064 for 680ee126-cfee-11ee-95bb-87774f69a715. 2024-02-20T12:49:48.776 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:49:48.746+0000 7fe22b7c4700 1 Processor -- start 2024-02-20T12:49:48.776 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:49:48.746+0000 7fe22b7c4700 1 -- start start 2024-02-20T12:49:48.776 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:49:48.746+0000 7fe22b7c4700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe224027610 0x7fe224101040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-02-20T12:49:48.776 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:49:48.746+0000 7fe22b7c4700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2240fca30 con 0x7fe224027610 2024-02-20T12:49:48.777 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:49:48.747+0000 7fe229560700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe224027610 0x7fe224101040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:49:48.777 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:49:48.747+0000 7fe229560700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe224027610 0x7fe224101040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44490/0 (socket says 172.21.15.64:44490) 2024-02-20T12:49:48.777 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:49:48.747+0000 7fe229560700 1 -- 172.21.15.64:0/413600685 learned_addr learned my addr 172.21.15.64:0/413600685 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-20T12:49:48.777 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:49:48.747+0000 7fe229560700 1 -- 172.21.15.64:0/413600685 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe224101530 con 0x7fe224027610 2024-02-20T12:49:48.777 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:49:48.747+0000 7fe229560700 1 --2- 172.21.15.64:0/413600685 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe224027610 0x7fe224101040 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 rx=0x7fe21800b5f0 tx=0x7fe218006ee0).ready entity=mon.0 client_cookie=f3109bc8c288ae90 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:49:48.777 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:49:48.748+0000 7fe217fff700 1 -- 172.21.15.64:0/413600685 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe21800d600 con 0x7fe224027610 2024-02-20T12:49:48.777 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:49:48.748+0000 7fe217fff700 1 -- 172.21.15.64:0/413600685 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe21800d760 con 0x7fe224027610 2024-02-20T12:49:48.777 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:49:48.748+0000 7fe217fff700 1 -- 172.21.15.64:0/413600685 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe21800da30 con 0x7fe224027610 2024-02-20T12:49:48.777 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:49:48.748+0000 7fe22b7c4700 1 -- 172.21.15.64:0/413600685 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe224027610 msgr2=0x7fe224101040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:49:48.777 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:49:48.748+0000 7fe22b7c4700 1 --2- 172.21.15.64:0/413600685 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe224027610 0x7fe224101040 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 rx=0x7fe21800b5f0 tx=0x7fe218006ee0).stop 2024-02-20T12:49:48.777 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:49:48.748+0000 7fe22b7c4700 1 -- 172.21.15.64:0/413600685 shutdown_connections 2024-02-20T12:49:48.777 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:49:48.748+0000 7fe22b7c4700 1 --2- 172.21.15.64:0/413600685 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe224027610 0x7fe224101040 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:49:48.777 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:49:48.748+0000 7fe22b7c4700 1 -- 172.21.15.64:0/413600685 >> 172.21.15.64:0/413600685 conn(0x7fe2240f6680 msgr2=0x7fe2240f8aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:49:48.778 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:49:48.748+0000 7fe22b7c4700 1 -- 172.21.15.64:0/413600685 shutdown_connections 2024-02-20T12:49:48.778 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:49:48.748+0000 7fe22b7c4700 1 -- 172.21.15.64:0/413600685 wait complete. 2024-02-20T12:49:48.778 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:49:48.749+0000 7fe22b7c4700 1 Processor -- start 2024-02-20T12:49:48.778 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:49:48.749+0000 7fe22b7c4700 1 -- start start 2024-02-20T12:49:48.778 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:49:48.749+0000 7fe22b7c4700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe224027610 0x7fe22410c750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:49:48.778 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:49:48.749+0000 7fe22b7c4700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe22410cc40 con 0x7fe224027610 2024-02-20T12:49:48.778 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:49:48.749+0000 7fe229560700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe224027610 0x7fe22410c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:49:48.778 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:49:48.749+0000 7fe229560700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe224027610 0x7fe22410c750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44506/0 (socket says 172.21.15.64:44506) 2024-02-20T12:49:48.778 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:49:48.749+0000 7fe229560700 1 -- 172.21.15.64:0/2777680399 learned_addr learned my addr 172.21.15.64:0/2777680399 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-20T12:49:48.778 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:49:48.749+0000 7fe229560700 1 -- 172.21.15.64:0/2777680399 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe21800b200 con 0x7fe224027610 2024-02-20T12:49:48.778 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:49:48.749+0000 7fe229560700 1 --2- 172.21.15.64:0/2777680399 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe224027610 0x7fe22410c750 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 rx=0x7fe218006c70 tx=0x7fe218016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:49:48.778 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:49:48.750+0000 7fe215ffb700 1 -- 172.21.15.64:0/2777680399 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe218011b40 con 0x7fe224027610 2024-02-20T12:49:48.778 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:49:48.750+0000 7fe215ffb700 1 -- 172.21.15.64:0/2777680399 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe218011ca0 con 0x7fe224027610 2024-02-20T12:49:48.778 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:49:48.750+0000 7fe22b7c4700 1 -- 172.21.15.64:0/2777680399 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe22410ce40 con 0x7fe224027610 2024-02-20T12:49:48.778 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:49:48.750+0000 7fe215ffb700 1 -- 172.21.15.64:0/2777680399 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe21801d430 con 0x7fe224027610 2024-02-20T12:49:48.779 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:49:48.750+0000 7fe22b7c4700 1 -- 172.21.15.64:0/2777680399 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe22410d2d0 con 0x7fe224027610 2024-02-20T12:49:48.779 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:49:48.750+0000 7fe215ffb700 1 -- 172.21.15.64:0/2777680399 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 1) v1 ==== 491+0+0 (secure 0 0 0) 0x7fe21800f070 con 0x7fe224027610 2024-02-20T12:49:48.779 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:49:48.750+0000 7fe215ffb700 1 -- 172.21.15.64:0/2777680399 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7fe21801dbf0 con 0x7fe224027610 2024-02-20T12:49:48.779 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:49:48.751+0000 7fe22b7c4700 1 -- 172.21.15.64:0/2777680399 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe224045bb0 con 0x7fe224027610 2024-02-20T12:49:48.779 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:49:48.753+0000 7fe215ffb700 1 -- 172.21.15.64:0/2777680399 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72097 (secure 0 0 0) 0x7fe218016070 con 0x7fe224027610 2024-02-20T12:49:48.826 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:49:48.822+0000 7fe22b7c4700 1 -- 172.21.15.64:0/2777680399 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "status"} v 0) v1 -- 0x7fe22410e7e0 con 0x7fe224027610 2024-02-20T12:49:48.826 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:49:48.823+0000 7fe215ffb700 1 -- 172.21.15.64:0/2777680399 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+325 (secure 0 0 0) 0x7fe218036070 con 0x7fe224027610 2024-02-20T12:49:48.826 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: cluster: 2024-02-20T12:49:48.826 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: id: 680ee126-cfee-11ee-95bb-87774f69a715 2024-02-20T12:49:48.826 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: health: HEALTH_OK 2024-02-20T12:49:48.827 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:49:48.827 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: services: 2024-02-20T12:49:48.827 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: mon: 1 daemons, quorum smithi064 (age 0.644715s) 2024-02-20T12:49:48.827 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: mgr: no daemons active 2024-02-20T12:49:48.827 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: osd: 0 osds: 0 up, 0 in 2024-02-20T12:49:48.827 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:49:48.827 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: data: 2024-02-20T12:49:48.827 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: pools: 0 pools, 0 pgs 2024-02-20T12:49:48.827 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: objects: 0 objects, 0 B 2024-02-20T12:49:48.827 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: usage: 0 B used, 0 B / 0 B avail 2024-02-20T12:49:48.827 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: pgs: 2024-02-20T12:49:48.827 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:49:48.827 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:49:48.823+0000 7fe22b7c4700 1 -- 172.21.15.64:0/2777680399 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe224027610 msgr2=0x7fe22410c750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:49:48.827 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:49:48.823+0000 7fe22b7c4700 1 --2- 172.21.15.64:0/2777680399 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe224027610 0x7fe22410c750 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 rx=0x7fe218006c70 tx=0x7fe218016000).stop 2024-02-20T12:49:48.828 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:49:48.824+0000 7fe22b7c4700 1 -- 172.21.15.64:0/2777680399 shutdown_connections 2024-02-20T12:49:48.828 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:49:48.824+0000 7fe22b7c4700 1 --2- 172.21.15.64:0/2777680399 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe224027610 0x7fe22410c750 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:49:48.828 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:49:48.824+0000 7fe22b7c4700 1 -- 172.21.15.64:0/2777680399 >> 172.21.15.64:0/2777680399 conn(0x7fe2240f6680 msgr2=0x7fe2240f8370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:49:48.828 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:49:48.824+0000 7fe22b7c4700 1 -- 172.21.15.64:0/2777680399 shutdown_connections 2024-02-20T12:49:48.828 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:49:48.824+0000 7fe22b7c4700 1 -- 172.21.15.64:0/2777680399 wait complete. 2024-02-20T12:49:49.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:49:49 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[37847]: cluster 2024-02-20T12:49:48.179379+0000 mon.smithi064 (mon.0) 2 : cluster [INF] mon.smithi064 is new leader, mons smithi064 in quorum (ranks 0) 2024-02-20T12:49:49.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:49:49 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[37847]: cluster 2024-02-20T12:49:48.179559+0000 mon.smithi064 (mon.0) 3 : cluster [DBG] monmap e1: 1 mons at {smithi064=[v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0]} 2024-02-20T12:49:49.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:49:49 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[37847]: cluster 2024-02-20T12:49:48.182981+0000 mon.smithi064 (mon.0) 4 : cluster [DBG] fsmap 2024-02-20T12:49:49.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:49:49 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[37847]: cluster 2024-02-20T12:49:48.186502+0000 mon.smithi064 (mon.0) 5 : cluster [DBG] osdmap e1: 0 total, 0 up, 0 in 2024-02-20T12:49:49.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:49:49 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[37847]: cluster 2024-02-20T12:49:48.186991+0000 mon.smithi064 (mon.0) 6 : cluster [DBG] mgrmap e1: no daemons active 2024-02-20T12:49:49.563 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:49:49 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[37847]: audit 2024-02-20T12:49:48.824054+0000 mon.smithi064 (mon.0) 7 : audit [DBG] from='client.? 172.21.15.64:0/2777680399' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-02-20T12:49:49.637 INFO:teuthology.orchestra.run.smithi064.stderr:mon is available 2024-02-20T12:49:49.637 INFO:teuthology.orchestra.run.smithi064.stderr:Assimilating anything we can from ceph.conf... 2024-02-20T12:49:50.192 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:49:50.185+0000 7f173ecdb700 1 Processor -- start 2024-02-20T12:49:50.192 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:49:50.185+0000 7f173ecdb700 1 -- start start 2024-02-20T12:49:50.192 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:49:50.185+0000 7f173ecdb700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1738027650 0x7f17380fec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-02-20T12:49:50.192 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:49:50.185+0000 7f173ecdb700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17380ff0f0 con 0x7f1738027650 2024-02-20T12:49:50.193 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:49:50.186+0000 7f173ca77700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1738027650 0x7f17380fec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:49:50.193 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:49:50.186+0000 7f173ca77700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1738027650 0x7f17380fec00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44520/0 (socket says 172.21.15.64:44520) 2024-02-20T12:49:50.193 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:49:50.186+0000 7f173ca77700 1 -- 172.21.15.64:0/2944957810 learned_addr learned my addr 172.21.15.64:0/2944957810 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-20T12:49:50.193 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:49:50.186+0000 7f173ca77700 1 -- 172.21.15.64:0/2944957810 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17380ff230 con 0x7f1738027650 2024-02-20T12:49:50.193 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:49:50.186+0000 7f173ca77700 1 --2- 172.21.15.64:0/2944957810 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1738027650 0x7f17380fec00 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 rx=0x7f172c00b5f0 tx=0x7f172c006ee0).ready entity=mon.0 client_cookie=84ae02616586a6ad server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:49:50.193 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:49:50.186+0000 7f17377fe700 1 -- 172.21.15.64:0/2944957810 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f172c00d600 con 0x7f1738027650 2024-02-20T12:49:50.193 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:49:50.186+0000 7f17377fe700 1 -- 172.21.15.64:0/2944957810 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f172c00d760 con 0x7f1738027650 2024-02-20T12:49:50.193 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:49:50.186+0000 7f17377fe700 1 -- 172.21.15.64:0/2944957810 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f172c00da30 con 0x7f1738027650 2024-02-20T12:49:50.193 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:49:50.187+0000 7f173ecdb700 1 -- 172.21.15.64:0/2944957810 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1738027650 msgr2=0x7f17380fec00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:49:50.193 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:49:50.187+0000 7f173ecdb700 1 --2- 172.21.15.64:0/2944957810 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1738027650 0x7f17380fec00 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 rx=0x7f172c00b5f0 tx=0x7f172c006ee0).stop 2024-02-20T12:49:50.194 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:49:50.187+0000 7f173ecdb700 1 -- 172.21.15.64:0/2944957810 shutdown_connections 2024-02-20T12:49:50.194 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:49:50.187+0000 7f173ecdb700 1 --2- 172.21.15.64:0/2944957810 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1738027650 0x7f17380fec00 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:49:50.194 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:49:50.187+0000 7f173ecdb700 1 -- 172.21.15.64:0/2944957810 >> 172.21.15.64:0/2944957810 conn(0x7f17380f6680 msgr2=0x7f17380f8aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:49:50.194 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:49:50.187+0000 7f173ecdb700 1 -- 172.21.15.64:0/2944957810 shutdown_connections 2024-02-20T12:49:50.194 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:49:50.187+0000 7f173ecdb700 1 -- 172.21.15.64:0/2944957810 wait complete. 2024-02-20T12:49:50.194 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:49:50.187+0000 7f173ecdb700 1 Processor -- start 2024-02-20T12:49:50.194 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:49:50.187+0000 7f173ecdb700 1 -- start start 2024-02-20T12:49:50.194 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:49:50.188+0000 7f173ecdb700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1738027650 0x7f173810a570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:49:50.194 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:49:50.188+0000 7f173ecdb700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f173810aa60 con 0x7f1738027650 2024-02-20T12:49:50.195 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:49:50.188+0000 7f173ca77700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1738027650 0x7f173810a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:49:50.195 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:49:50.188+0000 7f173ca77700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1738027650 0x7f173810a570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44530/0 (socket says 172.21.15.64:44530) 2024-02-20T12:49:50.195 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:49:50.188+0000 7f173ca77700 1 -- 172.21.15.64:0/1208918671 learned_addr learned my addr 172.21.15.64:0/1208918671 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-20T12:49:50.195 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:49:50.188+0000 7f173ca77700 1 -- 172.21.15.64:0/1208918671 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f172c00b200 con 0x7f1738027650 2024-02-20T12:49:50.195 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:49:50.188+0000 7f173ca77700 1 --2- 172.21.15.64:0/1208918671 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1738027650 0x7f173810a570 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 rx=0x7f172c006c70 tx=0x7f172c016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:49:50.195 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:49:50.188+0000 7f17357fa700 1 -- 172.21.15.64:0/1208918671 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f172c011b40 con 0x7f1738027650 2024-02-20T12:49:50.195 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:49:50.188+0000 7f17357fa700 1 -- 172.21.15.64:0/1208918671 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f172c011ca0 con 0x7f1738027650 2024-02-20T12:49:50.196 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:49:50.189+0000 7f173ecdb700 1 -- 172.21.15.64:0/1208918671 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f173810ac60 con 0x7f1738027650 2024-02-20T12:49:50.196 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:49:50.189+0000 7f17357fa700 1 -- 172.21.15.64:0/1208918671 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f172c01d430 con 0x7f1738027650 2024-02-20T12:49:50.196 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:49:50.189+0000 7f173ecdb700 1 -- 172.21.15.64:0/1208918671 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f173810b0f0 con 0x7f1738027650 2024-02-20T12:49:50.196 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:49:50.189+0000 7f17357fa700 1 -- 172.21.15.64:0/1208918671 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 1) v1 ==== 491+0+0 (secure 0 0 0) 0x7f172c00f070 con 0x7f1738027650 2024-02-20T12:49:50.196 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:49:50.189+0000 7f17357fa700 1 -- 172.21.15.64:0/1208918671 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f172c01dbf0 con 0x7f1738027650 2024-02-20T12:49:50.196 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:49:50.190+0000 7f173ecdb700 1 -- 172.21.15.64:0/1208918671 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1738045bb0 con 0x7f1738027650 2024-02-20T12:49:50.196 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:49:50.192+0000 7f17357fa700 1 -- 172.21.15.64:0/1208918671 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72097 (secure 0 0 0) 0x7f172c016070 con 0x7f1738027650 2024-02-20T12:49:50.259 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:49:50.258+0000 7f173ecdb700 1 -- 172.21.15.64:0/1208918671 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7f173810c600 con 0x7f1738027650 2024-02-20T12:49:50.266 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:49:50.261+0000 7f17357fa700 1 -- 172.21.15.64:0/1208918671 <== mon.0 v2:172.21.15.64:3300/0 7 ==== config(25 keys) v1 ==== 997+0+0 (secure 0 0 0) 0x7f172c01d5e0 con 0x7f1738027650 2024-02-20T12:49:50.266 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:49:50.261+0000 7f17357fa700 1 -- 172.21.15.64:0/1208918671 <== mon.0 v2:172.21.15.64:3300/0 8 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) v1 ==== 70+0+454 (secure 0 0 0) 0x7f172c036020 con 0x7f1738027650 2024-02-20T12:49:50.266 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:49:50.266 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: [global] 2024-02-20T12:49:50.266 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: fsid = 680ee126-cfee-11ee-95bb-87774f69a715 2024-02-20T12:49:50.267 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: log_to_journald = false 2024-02-20T12:49:50.267 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: mon_host = [v2:172.21.15.64:3300,v1:172.21.15.64:6789] 2024-02-20T12:49:50.267 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: mon_osd_allow_pg_remap = true 2024-02-20T12:49:50.267 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: mon_osd_allow_primary_affinity = true 2024-02-20T12:49:50.267 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: mon_warn_on_no_sortbitwise = false 2024-02-20T12:49:50.267 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: osd_crush_chooseleaf_type = 0 2024-02-20T12:49:50.267 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:49:50.267 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: [mgr] 2024-02-20T12:49:50.267 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: mgr/telemetry/nag = false 2024-02-20T12:49:50.267 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:49:50.267 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: [osd] 2024-02-20T12:49:50.268 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: mon_osd_backfillfull_ratio = 0.85 2024-02-20T12:49:50.268 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: mon_osd_full_ratio = 0.9 2024-02-20T12:49:50.268 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: mon_osd_nearfull_ratio = 0.8 2024-02-20T12:49:50.268 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: osd_map_max_advance = 10 2024-02-20T12:49:50.268 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: osd_sloppy_crc = true 2024-02-20T12:49:50.268 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:49:50.262+0000 7f173ecdb700 1 -- 172.21.15.64:0/1208918671 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1738027650 msgr2=0x7f173810a570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:49:50.268 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:49:50.262+0000 7f173ecdb700 1 --2- 172.21.15.64:0/1208918671 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1738027650 0x7f173810a570 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 rx=0x7f172c006c70 tx=0x7f172c016000).stop 2024-02-20T12:49:50.268 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:49:50.262+0000 7f173ecdb700 1 -- 172.21.15.64:0/1208918671 shutdown_connections 2024-02-20T12:49:50.269 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:49:50.262+0000 7f173ecdb700 1 --2- 172.21.15.64:0/1208918671 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1738027650 0x7f173810a570 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:49:50.269 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:49:50.262+0000 7f173ecdb700 1 -- 172.21.15.64:0/1208918671 >> 172.21.15.64:0/1208918671 conn(0x7f17380f6680 msgr2=0x7f17380f83e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:49:50.269 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:49:50.263+0000 7f173ecdb700 1 -- 172.21.15.64:0/1208918671 shutdown_connections 2024-02-20T12:49:50.269 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:49:50.263+0000 7f173ecdb700 1 -- 172.21.15.64:0/1208918671 wait complete. 2024-02-20T12:49:50.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:49:50 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[37847]: audit 2024-02-20T12:49:50.259871+0000 mon.smithi064 (mon.0) 8 : audit [INF] from='client.? 172.21.15.64:0/1208918671' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-02-20T12:49:50.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:49:50 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[37847]: audit 2024-02-20T12:49:50.262838+0000 mon.smithi064 (mon.0) 9 : audit [INF] from='client.? 172.21.15.64:0/1208918671' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-02-20T12:49:50.937 INFO:teuthology.orchestra.run.smithi064.stderr:Generating new minimal ceph.conf... 2024-02-20T12:49:52.773 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:49:52.483+0000 7f2836789700 1 Processor -- start 2024-02-20T12:49:52.773 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:49:52.483+0000 7f2836789700 1 -- start start 2024-02-20T12:49:52.774 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:49:52.484+0000 7f2836789700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2830027670 0x7f28300febe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-02-20T12:49:52.774 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:49:52.484+0000 7f2836789700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28300ff0d0 con 0x7f2830027670 2024-02-20T12:49:52.774 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:49:52.484+0000 7f282ffff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2830027670 0x7f28300febe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:49:52.774 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:49:52.484+0000 7f282ffff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2830027670 0x7f28300febe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54078/0 (socket says 172.21.15.64:54078) 2024-02-20T12:49:52.774 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:49:52.484+0000 7f282ffff700 1 -- 172.21.15.64:0/2108505994 learned_addr learned my addr 172.21.15.64:0/2108505994 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-20T12:49:52.774 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:49:52.484+0000 7f282ffff700 1 -- 172.21.15.64:0/2108505994 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28300ff210 con 0x7f2830027670 2024-02-20T12:49:52.774 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:49:52.484+0000 7f282ffff700 1 --2- 172.21.15.64:0/2108505994 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2830027670 0x7f28300febe0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 rx=0x7f282000b5f0 tx=0x7f2820006ee0).ready entity=mon.0 client_cookie=11c3fc236cad60a3 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:49:52.774 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:49:52.484+0000 7f282effd700 1 -- 172.21.15.64:0/2108505994 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f282000d600 con 0x7f2830027670 2024-02-20T12:49:52.774 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:49:52.484+0000 7f282effd700 1 -- 172.21.15.64:0/2108505994 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(25 keys) v1 ==== 997+0+0 (secure 0 0 0) 0x7f282000dbc0 con 0x7f2830027670 2024-02-20T12:49:52.774 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:49:52.484+0000 7f282effd700 1 -- 172.21.15.64:0/2108505994 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2820011b90 con 0x7f2830027670 2024-02-20T12:49:52.774 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:49:52.485+0000 7f2836789700 1 -- 172.21.15.64:0/2108505994 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2830027670 msgr2=0x7f28300febe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:49:52.774 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:49:52.485+0000 7f2836789700 1 --2- 172.21.15.64:0/2108505994 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2830027670 0x7f28300febe0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 rx=0x7f282000b5f0 tx=0x7f2820006ee0).stop 2024-02-20T12:49:52.774 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:49:52.485+0000 7f2836789700 1 -- 172.21.15.64:0/2108505994 shutdown_connections 2024-02-20T12:49:52.775 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:49:52.485+0000 7f2836789700 1 --2- 172.21.15.64:0/2108505994 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2830027670 0x7f28300febe0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:49:52.775 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:49:52.485+0000 7f2836789700 1 -- 172.21.15.64:0/2108505994 >> 172.21.15.64:0/2108505994 conn(0x7f28300f6680 msgr2=0x7f28300f8aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:49:52.775 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:49:52.485+0000 7f2836789700 1 -- 172.21.15.64:0/2108505994 shutdown_connections 2024-02-20T12:49:52.775 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:49:52.485+0000 7f2836789700 1 -- 172.21.15.64:0/2108505994 wait complete. 2024-02-20T12:49:52.775 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:49:52.486+0000 7f2836789700 1 Processor -- start 2024-02-20T12:49:52.775 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:49:52.486+0000 7f2836789700 1 -- start start 2024-02-20T12:49:52.775 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:49:52.486+0000 7f2836789700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2830027670 0x7f283010ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:49:52.775 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:49:52.486+0000 7f2836789700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f283010b220 con 0x7f2830027670 2024-02-20T12:49:52.775 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:49:52.486+0000 7f282ffff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2830027670 0x7f283010ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:49:52.775 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:49:52.486+0000 7f282ffff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2830027670 0x7f283010ad30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54080/0 (socket says 172.21.15.64:54080) 2024-02-20T12:49:52.775 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:49:52.486+0000 7f282ffff700 1 -- 172.21.15.64:0/1634254801 learned_addr learned my addr 172.21.15.64:0/1634254801 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-20T12:49:52.775 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:49:52.486+0000 7f282ffff700 1 -- 172.21.15.64:0/1634254801 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f282000b200 con 0x7f2830027670 2024-02-20T12:49:52.775 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:49:52.486+0000 7f282ffff700 1 --2- 172.21.15.64:0/1634254801 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2830027670 0x7f283010ad30 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 rx=0x7f2820000c00 tx=0x7f2820006ee0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:49:52.775 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:49:52.487+0000 7f282cff9700 1 -- 172.21.15.64:0/1634254801 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f282001b410 con 0x7f2830027670 2024-02-20T12:49:52.775 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:49:52.487+0000 7f282cff9700 1 -- 172.21.15.64:0/1634254801 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(25 keys) v1 ==== 997+0+0 (secure 0 0 0) 0x7f282001b9d0 con 0x7f2830027670 2024-02-20T12:49:52.776 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:49:52.487+0000 7f282cff9700 1 -- 172.21.15.64:0/1634254801 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f282001a620 con 0x7f2830027670 2024-02-20T12:49:52.776 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:49:52.487+0000 7f2836789700 1 -- 172.21.15.64:0/1634254801 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f283010b420 con 0x7f2830027670 2024-02-20T12:49:52.776 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:49:52.487+0000 7f2836789700 1 -- 172.21.15.64:0/1634254801 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f283010c030 con 0x7f2830027670 2024-02-20T12:49:52.776 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:49:52.487+0000 7f282cff9700 1 -- 172.21.15.64:0/1634254801 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 1) v1 ==== 491+0+0 (secure 0 0 0) 0x7f282002b070 con 0x7f2830027670 2024-02-20T12:49:52.776 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:49:52.487+0000 7f282cff9700 1 -- 172.21.15.64:0/1634254801 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f28200249b0 con 0x7f2830027670 2024-02-20T12:49:52.776 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:49:52.488+0000 7f2836789700 1 -- 172.21.15.64:0/1634254801 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28300591e0 con 0x7f2830027670 2024-02-20T12:49:52.776 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:49:52.490+0000 7f282cff9700 1 -- 172.21.15.64:0/1634254801 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72097 (secure 0 0 0) 0x7f282001f020 con 0x7f2830027670 2024-02-20T12:49:52.776 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:49:52.558+0000 7f2836789700 1 -- 172.21.15.64:0/1634254801 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7f28300fffb0 con 0x7f2830027670 2024-02-20T12:49:52.776 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:49:52.558+0000 7f282cff9700 1 -- 172.21.15.64:0/1634254801 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) v1 ==== 76+0+175 (secure 0 0 0) 0x7f2820016070 con 0x7f2830027670 2024-02-20T12:49:52.776 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:49:52.559+0000 7f2836789700 1 -- 172.21.15.64:0/1634254801 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2830027670 msgr2=0x7f283010ad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:49:52.776 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:49:52.559+0000 7f2836789700 1 --2- 172.21.15.64:0/1634254801 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2830027670 0x7f283010ad30 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 rx=0x7f2820000c00 tx=0x7f2820006ee0).stop 2024-02-20T12:49:52.776 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:49:52.559+0000 7f2836789700 1 -- 172.21.15.64:0/1634254801 shutdown_connections 2024-02-20T12:49:52.776 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:49:52.559+0000 7f2836789700 1 --2- 172.21.15.64:0/1634254801 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2830027670 0x7f283010ad30 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:49:52.776 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:49:52.559+0000 7f2836789700 1 -- 172.21.15.64:0/1634254801 >> 172.21.15.64:0/1634254801 conn(0x7f28300f6680 msgr2=0x7f28300f8180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:49:52.777 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:49:52.559+0000 7f2836789700 1 -- 172.21.15.64:0/1634254801 shutdown_connections 2024-02-20T12:49:52.777 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:49:52.559+0000 7f2836789700 1 -- 172.21.15.64:0/1634254801 wait complete. 2024-02-20T12:49:53.062 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:49:52 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[37847]: audit 2024-02-20T12:49:52.559210+0000 mon.smithi064 (mon.0) 10 : audit [DBG] from='client.? 172.21.15.64:0/1634254801' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-20T12:49:58.059 INFO:teuthology.orchestra.run.smithi064.stderr:Restarting the monitor... 2024-02-20T12:49:58.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:49:58 smithi064 systemd[1]: Stopping Ceph mon.smithi064 for 680ee126-cfee-11ee-95bb-87774f69a715... 2024-02-20T12:49:58.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:49:58 smithi064 bash[38176]: Error: no container with name or ID "ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon.smithi064" found: no such container 2024-02-20T12:49:59.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:49:59 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[37847]: debug 2024-02-20T12:49:59.063+0000 7fdcb7a3f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi064 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-stderr=true --default-log-stderr-prefix=debug --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-stderr=true (PID: 1) UID: 0 2024-02-20T12:49:59.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:49:59 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[37847]: debug 2024-02-20T12:49:59.063+0000 7fdcb7a3f700 -1 mon.smithi064@0(leader) e1 *** Got Signal Terminated *** 2024-02-20T12:50:02.812 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:02 smithi064 bash[38214]: ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064 2024-02-20T12:50:03.786 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 systemd[1]: ceph-680ee126-cfee-11ee-95bb-87774f69a715@mon.smithi064.service: Succeeded. 2024-02-20T12:50:03.786 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 systemd[1]: Stopped Ceph mon.smithi064 for 680ee126-cfee-11ee-95bb-87774f69a715. 2024-02-20T12:50:03.786 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 systemd[1]: Starting Ceph mon.smithi064 for 680ee126-cfee-11ee-95bb-87774f69a715... 2024-02-20T12:50:03.947 INFO:teuthology.orchestra.run.smithi064.stderr:Setting public_network to 172.21.0.0/20 in mon config section 2024-02-20T12:50:04.196 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: debug 2024-02-20T12:50:03.879+0000 7f9d3c5a8700 0 set uid:gid to 167:167 (ceph:ceph) 2024-02-20T12:50:04.196 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: debug 2024-02-20T12:50:03.879+0000 7f9d3c5a8700 0 ceph version 16.2.5 (0883bdea7337b95e4b611c768c0279868462204a) pacific (stable), process ceph-mon, pid 7 2024-02-20T12:50:04.196 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: debug 2024-02-20T12:50:03.879+0000 7f9d3c5a8700 0 pidfile_write: ignore empty --pid-file 2024-02-20T12:50:04.197 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: debug 2024-02-20T12:50:03.883+0000 7f9d3c5a8700 0 load: jerasure load: lrc load: isa 2024-02-20T12:50:04.197 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: debug 2024-02-20T12:50:03.884+0000 7f9d3c5a8700 4 rocksdb: RocksDB version: 6.8.1 2024-02-20T12:50:04.197 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: 2024-02-20T12:50:04.197 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: debug 2024-02-20T12:50:03.884+0000 7f9d3c5a8700 4 rocksdb: Git sha rocksdb_build_git_sha:@0@ 2024-02-20T12:50:04.197 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: debug 2024-02-20T12:50:03.884+0000 7f9d3c5a8700 4 rocksdb: Compile date Jul 8 2021 2024-02-20T12:50:04.197 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: debug 2024-02-20T12:50:03.884+0000 7f9d3c5a8700 4 rocksdb: DB SUMMARY 2024-02-20T12:50:04.197 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: 2024-02-20T12:50:04.197 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: debug 2024-02-20T12:50:03.884+0000 7f9d3c5a8700 4 rocksdb: CURRENT file: CURRENT 2024-02-20T12:50:04.197 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: 2024-02-20T12:50:04.197 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: debug 2024-02-20T12:50:03.884+0000 7f9d3c5a8700 4 rocksdb: IDENTITY file: IDENTITY 2024-02-20T12:50:04.197 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: 2024-02-20T12:50:04.197 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: debug 2024-02-20T12:50:03.884+0000 7f9d3c5a8700 4 rocksdb: MANIFEST file: MANIFEST-000005 size: 131 Bytes 2024-02-20T12:50:04.197 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: 2024-02-20T12:50:04.198 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: debug 2024-02-20T12:50:03.884+0000 7f9d3c5a8700 4 rocksdb: SST files in /var/lib/ceph/mon/ceph-smithi064/store.db dir, Total Num: 1, files: 000004.sst 2024-02-20T12:50:04.198 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: 2024-02-20T12:50:04.198 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: debug 2024-02-20T12:50:03.884+0000 7f9d3c5a8700 4 rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-smithi064/store.db: 000006.log size: 100974 ; 2024-02-20T12:50:04.198 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: 2024-02-20T12:50:04.198 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: debug 2024-02-20T12:50:03.884+0000 7f9d3c5a8700 4 rocksdb: Options.error_if_exists: 0 2024-02-20T12:50:04.198 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: debug 2024-02-20T12:50:03.884+0000 7f9d3c5a8700 4 rocksdb: Options.create_if_missing: 0 2024-02-20T12:50:04.198 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: debug 2024-02-20T12:50:03.884+0000 7f9d3c5a8700 4 rocksdb: Options.paranoid_checks: 1 2024-02-20T12:50:04.198 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: debug 2024-02-20T12:50:03.884+0000 7f9d3c5a8700 4 rocksdb: Options.env: 0x55edbaacf1c0 2024-02-20T12:50:04.198 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: debug 2024-02-20T12:50:03.884+0000 7f9d3c5a8700 4 rocksdb: Options.fs: Posix File System 2024-02-20T12:50:04.198 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: debug 2024-02-20T12:50:03.884+0000 7f9d3c5a8700 4 rocksdb: Options.info_log: 0x55edbbcc1a40 2024-02-20T12:50:04.198 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: debug 2024-02-20T12:50:03.884+0000 7f9d3c5a8700 4 rocksdb: Options.max_file_opening_threads: 16 2024-02-20T12:50:04.198 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: debug 2024-02-20T12:50:03.884+0000 7f9d3c5a8700 4 rocksdb: Options.statistics: (nil) 2024-02-20T12:50:04.198 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: debug 2024-02-20T12:50:03.884+0000 7f9d3c5a8700 4 rocksdb: Options.use_fsync: 0 2024-02-20T12:50:04.198 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: debug 2024-02-20T12:50:03.884+0000 7f9d3c5a8700 4 rocksdb: Options.max_log_file_size: 0 2024-02-20T12:50:04.199 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: debug 2024-02-20T12:50:03.884+0000 7f9d3c5a8700 4 rocksdb: Options.max_manifest_file_size: 1073741824 2024-02-20T12:50:04.199 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: debug 2024-02-20T12:50:03.884+0000 7f9d3c5a8700 4 rocksdb: Options.log_file_time_to_roll: 0 2024-02-20T12:50:04.199 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: debug 2024-02-20T12:50:03.884+0000 7f9d3c5a8700 4 rocksdb: Options.keep_log_file_num: 1000 2024-02-20T12:50:04.199 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: debug 2024-02-20T12:50:03.884+0000 7f9d3c5a8700 4 rocksdb: Options.recycle_log_file_num: 0 2024-02-20T12:50:04.199 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: debug 2024-02-20T12:50:03.884+0000 7f9d3c5a8700 4 rocksdb: Options.allow_fallocate: 1 2024-02-20T12:50:04.199 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: debug 2024-02-20T12:50:03.884+0000 7f9d3c5a8700 4 rocksdb: Options.allow_mmap_reads: 0 2024-02-20T12:50:04.199 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: debug 2024-02-20T12:50:03.884+0000 7f9d3c5a8700 4 rocksdb: Options.allow_mmap_writes: 0 2024-02-20T12:50:04.199 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: debug 2024-02-20T12:50:03.884+0000 7f9d3c5a8700 4 rocksdb: Options.use_direct_reads: 0 2024-02-20T12:50:04.199 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: debug 2024-02-20T12:50:03.884+0000 7f9d3c5a8700 4 rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-02-20T12:50:04.199 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: debug 2024-02-20T12:50:03.884+0000 7f9d3c5a8700 4 rocksdb: Options.create_missing_column_families: 0 2024-02-20T12:50:04.199 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: debug 2024-02-20T12:50:03.884+0000 7f9d3c5a8700 4 rocksdb: Options.db_log_dir: 2024-02-20T12:50:04.199 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: debug 2024-02-20T12:50:03.884+0000 7f9d3c5a8700 4 rocksdb: Options.wal_dir: /var/lib/ceph/mon/ceph-smithi064/store.db 2024-02-20T12:50:04.199 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: debug 2024-02-20T12:50:03.884+0000 7f9d3c5a8700 4 rocksdb: Options.table_cache_numshardbits: 6 2024-02-20T12:50:04.199 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: debug 2024-02-20T12:50:03.884+0000 7f9d3c5a8700 4 rocksdb: Options.max_subcompactions: 1 2024-02-20T12:50:04.200 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: debug 2024-02-20T12:50:03.884+0000 7f9d3c5a8700 4 rocksdb: Options.max_background_flushes: -1 2024-02-20T12:50:04.200 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: debug 2024-02-20T12:50:03.884+0000 7f9d3c5a8700 4 rocksdb: Options.WAL_ttl_seconds: 0 2024-02-20T12:50:04.200 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: debug 2024-02-20T12:50:03.884+0000 7f9d3c5a8700 4 rocksdb: Options.WAL_size_limit_MB: 0 2024-02-20T12:50:04.200 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: debug 2024-02-20T12:50:03.884+0000 7f9d3c5a8700 4 rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-02-20T12:50:04.200 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: debug 2024-02-20T12:50:03.884+0000 7f9d3c5a8700 4 rocksdb: Options.manifest_preallocation_size: 4194304 2024-02-20T12:50:04.200 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: debug 2024-02-20T12:50:03.884+0000 7f9d3c5a8700 4 rocksdb: Options.is_fd_close_on_exec: 1 2024-02-20T12:50:04.200 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: debug 2024-02-20T12:50:03.884+0000 7f9d3c5a8700 4 rocksdb: Options.advise_random_on_open: 1 2024-02-20T12:50:04.200 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: debug 2024-02-20T12:50:03.884+0000 7f9d3c5a8700 4 rocksdb: Options.db_write_buffer_size: 0 2024-02-20T12:50:04.200 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: debug 2024-02-20T12:50:03.884+0000 7f9d3c5a8700 4 rocksdb: Options.write_buffer_manager: 0x55edbc938030 2024-02-20T12:50:04.200 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: debug 2024-02-20T12:50:03.884+0000 7f9d3c5a8700 4 rocksdb: Options.access_hint_on_compaction_start: 1 2024-02-20T12:50:04.200 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: debug 2024-02-20T12:50:03.884+0000 7f9d3c5a8700 4 rocksdb: Options.new_table_reader_for_compaction_inputs: 0 2024-02-20T12:50:04.200 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: debug 2024-02-20T12:50:03.884+0000 7f9d3c5a8700 4 rocksdb: Options.random_access_max_buffer_size: 1048576 2024-02-20T12:50:04.201 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: debug 2024-02-20T12:50:03.884+0000 7f9d3c5a8700 4 rocksdb: Options.use_adaptive_mutex: 0 2024-02-20T12:50:04.201 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: debug 2024-02-20T12:50:03.884+0000 7f9d3c5a8700 4 rocksdb: Options.rate_limiter: (nil) 2024-02-20T12:50:04.201 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: debug 2024-02-20T12:50:03.884+0000 7f9d3c5a8700 4 rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-02-20T12:50:04.201 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: debug 2024-02-20T12:50:03.884+0000 7f9d3c5a8700 4 rocksdb: Options.wal_recovery_mode: 2 2024-02-20T12:50:04.201 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: debug 2024-02-20T12:50:03.884+0000 7f9d3c5a8700 4 rocksdb: Options.enable_thread_tracking: 0 2024-02-20T12:50:04.201 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: debug 2024-02-20T12:50:03.884+0000 7f9d3c5a8700 4 rocksdb: Options.enable_pipelined_write: 0 2024-02-20T12:50:04.201 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: debug 2024-02-20T12:50:03.884+0000 7f9d3c5a8700 4 rocksdb: Options.unordered_write: 0 2024-02-20T12:50:04.201 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: debug 2024-02-20T12:50:03.884+0000 7f9d3c5a8700 4 rocksdb: Options.allow_concurrent_memtable_write: 1 2024-02-20T12:50:04.203 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: debug 2024-02-20T12:50:03.884+0000 7f9d3c5a8700 4 rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-02-20T12:50:04.203 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: debug 2024-02-20T12:50:03.884+0000 7f9d3c5a8700 4 rocksdb: Options.write_thread_max_yield_usec: 100 2024-02-20T12:50:04.204 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: debug 2024-02-20T12:50:03.884+0000 7f9d3c5a8700 4 rocksdb: Options.write_thread_slow_yield_usec: 3 2024-02-20T12:50:04.204 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: debug 2024-02-20T12:50:03.884+0000 7f9d3c5a8700 4 rocksdb: Options.row_cache: None 2024-02-20T12:50:04.204 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: debug 2024-02-20T12:50:03.884+0000 7f9d3c5a8700 4 rocksdb: Options.wal_filter: None 2024-02-20T12:50:04.204 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: debug 2024-02-20T12:50:03.884+0000 7f9d3c5a8700 4 rocksdb: Options.avoid_flush_during_recovery: 0 2024-02-20T12:50:04.204 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: debug 2024-02-20T12:50:03.884+0000 7f9d3c5a8700 4 rocksdb: Options.allow_ingest_behind: 0 2024-02-20T12:50:04.204 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: debug 2024-02-20T12:50:03.884+0000 7f9d3c5a8700 4 rocksdb: Options.preserve_deletes: 0 2024-02-20T12:50:04.204 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: debug 2024-02-20T12:50:03.884+0000 7f9d3c5a8700 4 rocksdb: Options.two_write_queues: 0 2024-02-20T12:50:04.204 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: debug 2024-02-20T12:50:03.884+0000 7f9d3c5a8700 4 rocksdb: Options.manual_wal_flush: 0 2024-02-20T12:50:04.204 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: debug 2024-02-20T12:50:03.884+0000 7f9d3c5a8700 4 rocksdb: Options.atomic_flush: 0 2024-02-20T12:50:04.204 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: debug 2024-02-20T12:50:03.884+0000 7f9d3c5a8700 4 rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-02-20T12:50:04.204 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: debug 2024-02-20T12:50:03.884+0000 7f9d3c5a8700 4 rocksdb: Options.persist_stats_to_disk: 0 2024-02-20T12:50:04.204 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: debug 2024-02-20T12:50:03.884+0000 7f9d3c5a8700 4 rocksdb: Options.write_dbid_to_manifest: 0 2024-02-20T12:50:04.204 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: debug 2024-02-20T12:50:03.884+0000 7f9d3c5a8700 4 rocksdb: Options.log_readahead_size: 0 2024-02-20T12:50:04.204 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: debug 2024-02-20T12:50:03.884+0000 7f9d3c5a8700 4 rocksdb: Options.sst_file_checksum_func: Unknown 2024-02-20T12:50:04.205 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: debug 2024-02-20T12:50:03.884+0000 7f9d3c5a8700 4 rocksdb: Options.max_background_jobs: 2 2024-02-20T12:50:04.205 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: debug 2024-02-20T12:50:03.884+0000 7f9d3c5a8700 4 rocksdb: Options.max_background_compactions: -1 2024-02-20T12:50:04.205 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: debug 2024-02-20T12:50:03.884+0000 7f9d3c5a8700 4 rocksdb: Options.avoid_flush_during_shutdown: 0 2024-02-20T12:50:04.205 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: debug 2024-02-20T12:50:03.884+0000 7f9d3c5a8700 4 rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-02-20T12:50:04.205 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: debug 2024-02-20T12:50:03.884+0000 7f9d3c5a8700 4 rocksdb: Options.delayed_write_rate : 16777216 2024-02-20T12:50:04.205 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: debug 2024-02-20T12:50:03.884+0000 7f9d3c5a8700 4 rocksdb: Options.max_total_wal_size: 0 2024-02-20T12:50:04.205 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: debug 2024-02-20T12:50:03.884+0000 7f9d3c5a8700 4 rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-02-20T12:50:04.205 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: debug 2024-02-20T12:50:03.884+0000 7f9d3c5a8700 4 rocksdb: Options.stats_dump_period_sec: 600 2024-02-20T12:50:04.205 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: debug 2024-02-20T12:50:03.884+0000 7f9d3c5a8700 4 rocksdb: Options.stats_persist_period_sec: 600 2024-02-20T12:50:04.205 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: debug 2024-02-20T12:50:03.884+0000 7f9d3c5a8700 4 rocksdb: Options.stats_history_buffer_size: 1048576 2024-02-20T12:50:04.205 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: debug 2024-02-20T12:50:03.884+0000 7f9d3c5a8700 4 rocksdb: Options.max_open_files: -1 2024-02-20T12:50:04.205 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: debug 2024-02-20T12:50:03.884+0000 7f9d3c5a8700 4 rocksdb: Options.bytes_per_sync: 0 2024-02-20T12:50:04.205 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: debug 2024-02-20T12:50:03.884+0000 7f9d3c5a8700 4 rocksdb: Options.wal_bytes_per_sync: 0 2024-02-20T12:50:04.205 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: debug 2024-02-20T12:50:03.884+0000 7f9d3c5a8700 4 rocksdb: Options.strict_bytes_per_sync: 0 2024-02-20T12:50:04.206 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: debug 2024-02-20T12:50:03.884+0000 7f9d3c5a8700 4 rocksdb: Options.compaction_readahead_size: 0 2024-02-20T12:50:04.206 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: debug 2024-02-20T12:50:03.884+0000 7f9d3c5a8700 4 rocksdb: Compression algorithms supported: 2024-02-20T12:50:04.206 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: debug 2024-02-20T12:50:03.884+0000 7f9d3c5a8700 4 rocksdb: kZSTDNotFinalCompression supported: 0 2024-02-20T12:50:04.206 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: debug 2024-02-20T12:50:03.884+0000 7f9d3c5a8700 4 rocksdb: kZSTD supported: 0 2024-02-20T12:50:04.206 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: debug 2024-02-20T12:50:03.884+0000 7f9d3c5a8700 4 rocksdb: kXpressCompression supported: 0 2024-02-20T12:50:04.206 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: debug 2024-02-20T12:50:03.884+0000 7f9d3c5a8700 4 rocksdb: kLZ4HCCompression supported: 1 2024-02-20T12:50:04.206 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: debug 2024-02-20T12:50:03.884+0000 7f9d3c5a8700 4 rocksdb: kLZ4Compression supported: 1 2024-02-20T12:50:04.206 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: debug 2024-02-20T12:50:03.884+0000 7f9d3c5a8700 4 rocksdb: kBZip2Compression supported: 0 2024-02-20T12:50:04.206 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: debug 2024-02-20T12:50:03.884+0000 7f9d3c5a8700 4 rocksdb: kZlibCompression supported: 1 2024-02-20T12:50:04.206 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: debug 2024-02-20T12:50:03.884+0000 7f9d3c5a8700 4 rocksdb: kSnappyCompression supported: 1 2024-02-20T12:50:04.206 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: debug 2024-02-20T12:50:03.884+0000 7f9d3c5a8700 4 rocksdb: Fast CRC32 supported: Supported on x86 2024-02-20T12:50:04.206 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: debug 2024-02-20T12:50:03.884+0000 7f9d3c5a8700 4 rocksdb: [version_set.cc:4413] Recovering from manifest file: /var/lib/ceph/mon/ceph-smithi064/store.db/MANIFEST-000005 2024-02-20T12:50:04.206 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: 2024-02-20T12:50:04.206 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: debug 2024-02-20T12:50:03.884+0000 7f9d3c5a8700 4 rocksdb: [column_family.cc:552] --------------- Options for column family [default]: 2024-02-20T12:50:04.206 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: 2024-02-20T12:50:04.206 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: debug 2024-02-20T12:50:03.884+0000 7f9d3c5a8700 4 rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-02-20T12:50:04.207 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: debug 2024-02-20T12:50:03.884+0000 7f9d3c5a8700 4 rocksdb: Options.merge_operator: 2024-02-20T12:50:04.207 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: debug 2024-02-20T12:50:03.884+0000 7f9d3c5a8700 4 rocksdb: Options.compaction_filter: None 2024-02-20T12:50:04.207 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: debug 2024-02-20T12:50:03.884+0000 7f9d3c5a8700 4 rocksdb: Options.compaction_filter_factory: None 2024-02-20T12:50:04.207 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: debug 2024-02-20T12:50:03.884+0000 7f9d3c5a8700 4 rocksdb: Options.memtable_factory: SkipListFactory 2024-02-20T12:50:04.207 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: debug 2024-02-20T12:50:03.884+0000 7f9d3c5a8700 4 rocksdb: Options.table_factory: BlockBasedTable 2024-02-20T12:50:04.207 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: debug 2024-02-20T12:50:03.884+0000 7f9d3c5a8700 4 rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55edbbbda170) 2024-02-20T12:50:04.207 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cache_index_and_filter_blocks: 1 2024-02-20T12:50:04.207 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cache_index_and_filter_blocks_with_high_priority: 0 2024-02-20T12:50:04.207 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: pin_l0_filter_and_index_blocks_in_cache: 0 2024-02-20T12:50:04.207 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: pin_top_level_index_and_filter: 1 2024-02-20T12:50:04.207 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: index_type: 0 2024-02-20T12:50:04.207 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: data_block_index_type: 0 2024-02-20T12:50:04.207 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: index_shortening: 1 2024-02-20T12:50:04.207 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: data_block_hash_table_util_ratio: 0.750000 2024-02-20T12:50:04.207 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: hash_index_allow_collision: 1 2024-02-20T12:50:04.207 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: checksum: 1 2024-02-20T12:50:04.208 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: no_block_cache: 0 2024-02-20T12:50:04.208 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: block_cache: 0x55edbbc10f10 2024-02-20T12:50:04.208 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: block_cache_name: BinnedLRUCache 2024-02-20T12:50:04.208 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: block_cache_options: 2024-02-20T12:50:04.208 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: capacity : 536870912 2024-02-20T12:50:04.208 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: num_shard_bits : 4 2024-02-20T12:50:04.208 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: strict_capacity_limit : 0 2024-02-20T12:50:04.208 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: high_pri_pool_ratio: 0.000 2024-02-20T12:50:04.208 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: block_cache_compressed: (nil) 2024-02-20T12:50:04.208 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: persistent_cache: (nil) 2024-02-20T12:50:04.208 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: block_size: 4096 2024-02-20T12:50:04.208 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: block_size_deviation: 10 2024-02-20T12:50:04.208 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: block_restart_interval: 16 2024-02-20T12:50:04.208 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: index_block_restart_interval: 1 2024-02-20T12:50:04.209 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: metadata_block_size: 4096 2024-02-20T12:50:04.209 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: partition_filters: 0 2024-02-20T12:50:04.209 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: use_delta_encoding: 1 2024-02-20T12:50:04.209 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: filter_policy: rocksdb.BuiltinBloomFilter 2024-02-20T12:50:04.209 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: whole_key_filtering: 1 2024-02-20T12:50:04.209 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: verify_compression: 0 2024-02-20T12:50:04.209 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: read_amp_bytes_per_bit: 0 2024-02-20T12:50:04.209 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: format_version: 2 2024-02-20T12:50:04.209 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: enable_index_compression: 1 2024-02-20T12:50:04.209 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: block_align: 0 2024-02-20T12:50:04.209 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: 2024-02-20T12:50:04.209 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: debug 2024-02-20T12:50:03.884+0000 7f9d3c5a8700 4 rocksdb: Options.write_buffer_size: 33554432 2024-02-20T12:50:04.209 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: debug 2024-02-20T12:50:03.884+0000 7f9d3c5a8700 4 rocksdb: Options.max_write_buffer_number: 2 2024-02-20T12:50:04.209 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: debug 2024-02-20T12:50:03.884+0000 7f9d3c5a8700 4 rocksdb: Options.compression: NoCompression 2024-02-20T12:50:04.209 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: debug 2024-02-20T12:50:03.884+0000 7f9d3c5a8700 4 rocksdb: Options.bottommost_compression: Disabled 2024-02-20T12:50:04.209 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: debug 2024-02-20T12:50:03.884+0000 7f9d3c5a8700 4 rocksdb: Options.prefix_extractor: nullptr 2024-02-20T12:50:04.210 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: debug 2024-02-20T12:50:03.884+0000 7f9d3c5a8700 4 rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-02-20T12:50:04.210 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: debug 2024-02-20T12:50:03.884+0000 7f9d3c5a8700 4 rocksdb: Options.num_levels: 7 2024-02-20T12:50:04.210 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: debug 2024-02-20T12:50:03.884+0000 7f9d3c5a8700 4 rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-02-20T12:50:04.210 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: debug 2024-02-20T12:50:03.884+0000 7f9d3c5a8700 4 rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-02-20T12:50:04.210 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: debug 2024-02-20T12:50:03.884+0000 7f9d3c5a8700 4 rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-02-20T12:50:04.210 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: debug 2024-02-20T12:50:03.884+0000 7f9d3c5a8700 4 rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-02-20T12:50:04.210 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: debug 2024-02-20T12:50:03.884+0000 7f9d3c5a8700 4 rocksdb: Options.bottommost_compression_opts.level: 32767 2024-02-20T12:50:04.210 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: debug 2024-02-20T12:50:03.884+0000 7f9d3c5a8700 4 rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-02-20T12:50:04.210 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: debug 2024-02-20T12:50:03.884+0000 7f9d3c5a8700 4 rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-02-20T12:50:04.210 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: debug 2024-02-20T12:50:03.884+0000 7f9d3c5a8700 4 rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-02-20T12:50:04.210 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: debug 2024-02-20T12:50:03.884+0000 7f9d3c5a8700 4 rocksdb: Options.bottommost_compression_opts.enabled: false 2024-02-20T12:50:04.210 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: debug 2024-02-20T12:50:03.884+0000 7f9d3c5a8700 4 rocksdb: Options.compression_opts.window_bits: -14 2024-02-20T12:50:04.210 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: debug 2024-02-20T12:50:03.884+0000 7f9d3c5a8700 4 rocksdb: Options.compression_opts.level: 32767 2024-02-20T12:50:04.210 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: debug 2024-02-20T12:50:03.884+0000 7f9d3c5a8700 4 rocksdb: Options.compression_opts.strategy: 0 2024-02-20T12:50:04.210 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: debug 2024-02-20T12:50:03.884+0000 7f9d3c5a8700 4 rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-02-20T12:50:04.211 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: debug 2024-02-20T12:50:03.884+0000 7f9d3c5a8700 4 rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-02-20T12:50:04.211 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: debug 2024-02-20T12:50:03.884+0000 7f9d3c5a8700 4 rocksdb: Options.compression_opts.enabled: false 2024-02-20T12:50:04.211 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: debug 2024-02-20T12:50:03.884+0000 7f9d3c5a8700 4 rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-02-20T12:50:04.211 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: debug 2024-02-20T12:50:03.884+0000 7f9d3c5a8700 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-02-20T12:50:04.211 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: debug 2024-02-20T12:50:03.884+0000 7f9d3c5a8700 4 rocksdb: Options.level0_stop_writes_trigger: 36 2024-02-20T12:50:04.211 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: debug 2024-02-20T12:50:03.884+0000 7f9d3c5a8700 4 rocksdb: Options.target_file_size_base: 67108864 2024-02-20T12:50:04.211 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: debug 2024-02-20T12:50:03.884+0000 7f9d3c5a8700 4 rocksdb: Options.target_file_size_multiplier: 1 2024-02-20T12:50:04.211 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: debug 2024-02-20T12:50:03.884+0000 7f9d3c5a8700 4 rocksdb: Options.max_bytes_for_level_base: 268435456 2024-02-20T12:50:04.211 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: debug 2024-02-20T12:50:03.884+0000 7f9d3c5a8700 4 rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-02-20T12:50:04.211 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: debug 2024-02-20T12:50:03.884+0000 7f9d3c5a8700 4 rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-02-20T12:50:04.211 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: debug 2024-02-20T12:50:03.884+0000 7f9d3c5a8700 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-02-20T12:50:04.211 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: debug 2024-02-20T12:50:03.884+0000 7f9d3c5a8700 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-02-20T12:50:04.211 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: debug 2024-02-20T12:50:03.884+0000 7f9d3c5a8700 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-02-20T12:50:04.211 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: debug 2024-02-20T12:50:03.884+0000 7f9d3c5a8700 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-02-20T12:50:04.212 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: debug 2024-02-20T12:50:03.884+0000 7f9d3c5a8700 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-02-20T12:50:04.212 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: debug 2024-02-20T12:50:03.884+0000 7f9d3c5a8700 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-02-20T12:50:04.212 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: debug 2024-02-20T12:50:03.884+0000 7f9d3c5a8700 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-02-20T12:50:04.212 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: debug 2024-02-20T12:50:03.884+0000 7f9d3c5a8700 4 rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-02-20T12:50:04.212 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: debug 2024-02-20T12:50:03.884+0000 7f9d3c5a8700 4 rocksdb: Options.max_compaction_bytes: 1677721600 2024-02-20T12:50:04.212 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: debug 2024-02-20T12:50:03.884+0000 7f9d3c5a8700 4 rocksdb: Options.arena_block_size: 4194304 2024-02-20T12:50:04.212 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: debug 2024-02-20T12:50:03.884+0000 7f9d3c5a8700 4 rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-02-20T12:50:04.212 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: debug 2024-02-20T12:50:03.884+0000 7f9d3c5a8700 4 rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-02-20T12:50:04.212 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: debug 2024-02-20T12:50:03.884+0000 7f9d3c5a8700 4 rocksdb: Options.rate_limit_delay_max_milliseconds: 100 2024-02-20T12:50:04.212 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: debug 2024-02-20T12:50:03.884+0000 7f9d3c5a8700 4 rocksdb: Options.disable_auto_compactions: 0 2024-02-20T12:50:04.212 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: debug 2024-02-20T12:50:03.884+0000 7f9d3c5a8700 4 rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-02-20T12:50:04.212 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: debug 2024-02-20T12:50:03.884+0000 7f9d3c5a8700 4 rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-02-20T12:50:04.212 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: debug 2024-02-20T12:50:03.884+0000 7f9d3c5a8700 4 rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-02-20T12:50:04.212 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: debug 2024-02-20T12:50:03.884+0000 7f9d3c5a8700 4 rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-02-20T12:50:04.212 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: debug 2024-02-20T12:50:03.884+0000 7f9d3c5a8700 4 rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-02-20T12:50:04.213 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: debug 2024-02-20T12:50:03.884+0000 7f9d3c5a8700 4 rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-02-20T12:50:04.213 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: debug 2024-02-20T12:50:03.884+0000 7f9d3c5a8700 4 rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-02-20T12:50:04.213 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: debug 2024-02-20T12:50:03.884+0000 7f9d3c5a8700 4 rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-02-20T12:50:04.213 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: debug 2024-02-20T12:50:03.884+0000 7f9d3c5a8700 4 rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-02-20T12:50:04.213 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: debug 2024-02-20T12:50:03.884+0000 7f9d3c5a8700 4 rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-02-20T12:50:04.213 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: debug 2024-02-20T12:50:03.884+0000 7f9d3c5a8700 4 rocksdb: Options.table_properties_collectors: 2024-02-20T12:50:04.213 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: debug 2024-02-20T12:50:03.884+0000 7f9d3c5a8700 4 rocksdb: Options.inplace_update_support: 0 2024-02-20T12:50:04.213 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: debug 2024-02-20T12:50:03.884+0000 7f9d3c5a8700 4 rocksdb: Options.inplace_update_num_locks: 10000 2024-02-20T12:50:04.213 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: debug 2024-02-20T12:50:03.884+0000 7f9d3c5a8700 4 rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-02-20T12:50:04.213 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: debug 2024-02-20T12:50:03.884+0000 7f9d3c5a8700 4 rocksdb: Options.memtable_whole_key_filtering: 0 2024-02-20T12:50:04.213 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: debug 2024-02-20T12:50:03.884+0000 7f9d3c5a8700 4 rocksdb: Options.memtable_huge_page_size: 0 2024-02-20T12:50:04.213 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: debug 2024-02-20T12:50:03.884+0000 7f9d3c5a8700 4 rocksdb: Options.bloom_locality: 0 2024-02-20T12:50:04.213 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: debug 2024-02-20T12:50:03.884+0000 7f9d3c5a8700 4 rocksdb: Options.max_successive_merges: 0 2024-02-20T12:50:04.213 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: debug 2024-02-20T12:50:03.884+0000 7f9d3c5a8700 4 rocksdb: Options.optimize_filters_for_hits: 0 2024-02-20T12:50:04.213 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: debug 2024-02-20T12:50:03.884+0000 7f9d3c5a8700 4 rocksdb: Options.paranoid_file_checks: 0 2024-02-20T12:50:04.214 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: debug 2024-02-20T12:50:03.884+0000 7f9d3c5a8700 4 rocksdb: Options.force_consistency_checks: 0 2024-02-20T12:50:04.214 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: debug 2024-02-20T12:50:03.884+0000 7f9d3c5a8700 4 rocksdb: Options.report_bg_io_stats: 0 2024-02-20T12:50:04.214 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: debug 2024-02-20T12:50:03.884+0000 7f9d3c5a8700 4 rocksdb: Options.ttl: 2592000 2024-02-20T12:50:04.214 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: debug 2024-02-20T12:50:03.884+0000 7f9d3c5a8700 4 rocksdb: Options.periodic_compaction_seconds: 0 2024-02-20T12:50:04.214 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: debug 2024-02-20T12:50:03.885+0000 7f9d3c5a8700 4 rocksdb: [version_set.cc:4568] Recovered from manifest file:/var/lib/ceph/mon/ceph-smithi064/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 5, log_number is 4,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-02-20T12:50:04.214 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: 2024-02-20T12:50:04.214 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: debug 2024-02-20T12:50:03.885+0000 7f9d3c5a8700 4 rocksdb: [version_set.cc:4577] Column family [default] (ID 0), log number is 4 2024-02-20T12:50:04.214 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: 2024-02-20T12:50:04.214 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: debug 2024-02-20T12:50:03.885+0000 7f9d3c5a8700 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1708433403886453, "job": 1, "event": "recovery_started", "log_files": [6]} 2024-02-20T12:50:04.214 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: debug 2024-02-20T12:50:03.885+0000 7f9d3c5a8700 4 rocksdb: [db_impl/db_impl_open.cc:760] Recovering log #6 mode 2 2024-02-20T12:50:04.214 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: debug 2024-02-20T12:50:03.885+0000 7f9d3c5a8700 3 rocksdb: [le/block_based/filter_policy.cc:584] Using legacy Bloom filter with high (20) bits/key. Dramatic filter space and/or accuracy improvement is available with format_version>=5. 2024-02-20T12:50:04.214 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: debug 2024-02-20T12:50:03.886+0000 7f9d3c5a8700 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1708433403887450, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 7, "file_size": 95866, "table_properties": {"data_size": 93961, "index_size": 382, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 709, "raw_key_size": 12950, "raw_average_key_size": 49, "raw_value_size": 86828, "raw_average_value_size": 331, "num_data_blocks": 12, "num_entries": 262, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "rocksdb.BuiltinBloomFilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; ", "creation_time": 1708433403, "oldest_key_time": 3, "file_creation_time": 0}} 2024-02-20T12:50:04.214 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: debug 2024-02-20T12:50:03.886+0000 7f9d3c5a8700 4 rocksdb: [version_set.cc:3826] Creating manifest 8 2024-02-20T12:50:04.214 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: 2024-02-20T12:50:04.215 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: debug 2024-02-20T12:50:03.886+0000 7f9d3c5a8700 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1708433403887874, "job": 1, "event": "recovery_finished"} 2024-02-20T12:50:04.215 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: debug 2024-02-20T12:50:03.887+0000 7f9d3c5a8700 4 rocksdb: DB pointer 0x55edbbcd5800 2024-02-20T12:50:04.215 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: debug 2024-02-20T12:50:03.887+0000 7f9d233b0700 4 rocksdb: [db_impl/db_impl.cc:850] ------- DUMPING STATS ------- 2024-02-20T12:50:04.215 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: debug 2024-02-20T12:50:03.887+0000 7f9d233b0700 4 rocksdb: [db_impl/db_impl.cc:851] 2024-02-20T12:50:04.215 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: ** DB Stats ** 2024-02-20T12:50:04.215 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: Uptime(secs): 0.0 total, 0.0 interval 2024-02-20T12:50:04.215 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-02-20T12:50:04.215 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-02-20T12:50:04.215 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-02-20T12:50:04.215 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-02-20T12:50:04.215 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 MB, 0.00 MB/s 2024-02-20T12:50:04.215 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-02-20T12:50:04.215 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: 2024-02-20T12:50:04.215 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: ** Compaction Stats [default] ** 2024-02-20T12:50:04.215 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-02-20T12:50:04.216 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-02-20T12:50:04.216 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: L0 2/0 95.31 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 143.1 0.00 0.00 1 0.001 0 0 2024-02-20T12:50:04.216 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: Sum 2/0 95.31 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 143.1 0.00 0.00 1 0.001 0 0 2024-02-20T12:50:04.216 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 143.1 0.00 0.00 1 0.001 0 0 2024-02-20T12:50:04.216 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: 2024-02-20T12:50:04.216 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: ** Compaction Stats [default] ** 2024-02-20T12:50:04.216 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-02-20T12:50:04.216 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-02-20T12:50:04.216 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 143.1 0.00 0.00 1 0.001 0 0 2024-02-20T12:50:04.216 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: Uptime(secs): 0.0 total, 0.0 interval 2024-02-20T12:50:04.216 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: Flush(GB): cumulative 0.000, interval 0.000 2024-02-20T12:50:04.216 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: AddFile(GB): cumulative 0.000, interval 0.000 2024-02-20T12:50:04.216 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: AddFile(Total Files): cumulative 0, interval 0 2024-02-20T12:50:04.216 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: AddFile(L0 Files): cumulative 0, interval 0 2024-02-20T12:50:04.216 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: AddFile(Keys): cumulative 0, interval 0 2024-02-20T12:50:04.216 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: Cumulative compaction: 0.00 GB write, 29.49 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-20T12:50:04.217 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: Interval compaction: 0.00 GB write, 29.49 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-20T12:50:04.217 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-02-20T12:50:04.217 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: 2024-02-20T12:50:04.217 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: ** File Read Latency Histogram By Level [default] ** 2024-02-20T12:50:04.217 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: 2024-02-20T12:50:04.217 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: ** Compaction Stats [default] ** 2024-02-20T12:50:04.217 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-02-20T12:50:04.217 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-02-20T12:50:04.217 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: L0 2/0 95.31 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 143.1 0.00 0.00 1 0.001 0 0 2024-02-20T12:50:04.217 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: Sum 2/0 95.31 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 143.1 0.00 0.00 1 0.001 0 0 2024-02-20T12:50:04.217 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 2024-02-20T12:50:04.217 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: 2024-02-20T12:50:04.217 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: ** Compaction Stats [default] ** 2024-02-20T12:50:04.217 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-02-20T12:50:04.217 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-02-20T12:50:04.217 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 143.1 0.00 0.00 1 0.001 0 0 2024-02-20T12:50:04.218 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: Uptime(secs): 0.0 total, 0.0 interval 2024-02-20T12:50:04.218 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: Flush(GB): cumulative 0.000, interval 0.000 2024-02-20T12:50:04.218 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: AddFile(GB): cumulative 0.000, interval 0.000 2024-02-20T12:50:04.218 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: AddFile(Total Files): cumulative 0, interval 0 2024-02-20T12:50:04.218 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: AddFile(L0 Files): cumulative 0, interval 0 2024-02-20T12:50:04.218 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: AddFile(Keys): cumulative 0, interval 0 2024-02-20T12:50:04.218 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: Cumulative compaction: 0.00 GB write, 28.96 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-20T12:50:04.218 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: Interval compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-20T12:50:04.218 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-02-20T12:50:04.218 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: 2024-02-20T12:50:04.218 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: ** File Read Latency Histogram By Level [default] ** 2024-02-20T12:50:04.218 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: 2024-02-20T12:50:04.218 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: debug 2024-02-20T12:50:03.888+0000 7f9d3c5a8700 0 starting mon.smithi064 rank 0 at public addrs [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] at bind addrs [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] mon_data /var/lib/ceph/mon/ceph-smithi064 fsid 680ee126-cfee-11ee-95bb-87774f69a715 2024-02-20T12:50:04.218 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: debug 2024-02-20T12:50:03.889+0000 7f9d3c5a8700 1 mon.smithi064@-1(???) e1 preinit fsid 680ee126-cfee-11ee-95bb-87774f69a715 2024-02-20T12:50:04.218 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: debug 2024-02-20T12:50:03.889+0000 7f9d3c5a8700 0 mon.smithi064@-1(???).mds e1 new map 2024-02-20T12:50:04.218 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: debug 2024-02-20T12:50:03.889+0000 7f9d3c5a8700 0 mon.smithi064@-1(???).mds e1 print_map 2024-02-20T12:50:04.218 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: e1 2024-02-20T12:50:04.219 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: enable_multiple, ever_enabled_multiple: 1,1 2024-02-20T12:50:04.219 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-20T12:50:04.219 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: legacy client fscid: -1 2024-02-20T12:50:04.219 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: 2024-02-20T12:50:04.219 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: No filesystems configured 2024-02-20T12:50:04.219 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: 2024-02-20T12:50:04.219 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: debug 2024-02-20T12:50:03.889+0000 7f9d3c5a8700 0 mon.smithi064@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-02-20T12:50:04.219 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: debug 2024-02-20T12:50:03.889+0000 7f9d3c5a8700 0 mon.smithi064@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-02-20T12:50:04.219 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: debug 2024-02-20T12:50:03.889+0000 7f9d3c5a8700 0 mon.smithi064@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-02-20T12:50:04.219 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: debug 2024-02-20T12:50:03.889+0000 7f9d3c5a8700 0 mon.smithi064@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-02-20T12:50:04.219 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:49:52.559210+0000 mon.smithi064 (mon.0) 10 : audit [DBG] from='client.? 172.21.15.64:0/1634254801' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-20T12:50:04.219 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: debug 2024-02-20T12:50:03.889+0000 7f9d3c5a8700 1 mon.smithi064@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-02-20T12:50:04.219 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 bash[38338]: 5b59c3a70e6d8f7318245afde601ab5b0799dead5b313f553d06468201df6421 2024-02-20T12:50:04.219 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 systemd[1]: Started Ceph mon.smithi064 for 680ee126-cfee-11ee-95bb-87774f69a715. 2024-02-20T12:50:04.219 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:50:03.905682+0000 mon.smithi064 (mon.0) 1 : cluster [INF] mon.smithi064 is new leader, mons smithi064 in quorum (ranks 0) 2024-02-20T12:50:04.219 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:50:03.905721+0000 mon.smithi064 (mon.0) 2 : cluster [DBG] monmap e1: 1 mons at {smithi064=[v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0]} 2024-02-20T12:50:04.220 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:50:03.905755+0000 mon.smithi064 (mon.0) 3 : cluster [DBG] fsmap 2024-02-20T12:50:04.220 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:50:03.905777+0000 mon.smithi064 (mon.0) 4 : cluster [DBG] osdmap e1: 0 total, 0 up, 0 in 2024-02-20T12:50:04.220 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:50:03.906132+0000 mon.smithi064 (mon.0) 5 : cluster [DBG] mgrmap e1: no daemons active 2024-02-20T12:50:04.492 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:04.491+0000 7fbe77dfe700 1 Processor -- start 2024-02-20T12:50:04.496 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:04.491+0000 7fbe77dfe700 1 -- start start 2024-02-20T12:50:04.496 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:04.492+0000 7fbe77dfe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbe700fc4d0 0x7fbe700fc8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-02-20T12:50:04.496 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:04.492+0000 7fbe77dfe700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe700fcde0 con 0x7fbe700fc4d0 2024-02-20T12:50:04.496 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:04.492+0000 7fbe75b9a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbe700fc4d0 0x7fbe700fc8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:50:04.496 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:04.492+0000 7fbe75b9a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbe700fc4d0 0x7fbe700fc8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:40940/0 (socket says 172.21.15.64:40940) 2024-02-20T12:50:04.496 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:04.492+0000 7fbe75b9a700 1 -- 172.21.15.64:0/2965355077 learned_addr learned my addr 172.21.15.64:0/2965355077 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-20T12:50:04.497 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:04.492+0000 7fbe75b9a700 1 -- 172.21.15.64:0/2965355077 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe700fcf20 con 0x7fbe700fc4d0 2024-02-20T12:50:04.497 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:04.492+0000 7fbe75b9a700 1 --2- 172.21.15.64:0/2965355077 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbe700fc4d0 0x7fbe700fc8f0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 rx=0x7fbe6000b5f0 tx=0x7fbe60006ee0).ready entity=mon.0 client_cookie=8a407326f19369d server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:50:04.497 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:04.493+0000 7fbe74b98700 1 -- 172.21.15.64:0/2965355077 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fbe6000d600 con 0x7fbe700fc4d0 2024-02-20T12:50:04.497 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:04.493+0000 7fbe74b98700 1 -- 172.21.15.64:0/2965355077 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(25 keys) v1 ==== 997+0+0 (secure 0 0 0) 0x7fbe6000dbc0 con 0x7fbe700fc4d0 2024-02-20T12:50:04.497 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:04.493+0000 7fbe74b98700 1 -- 172.21.15.64:0/2965355077 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fbe60011b90 con 0x7fbe700fc4d0 2024-02-20T12:50:04.497 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:04.493+0000 7fbe77dfe700 1 -- 172.21.15.64:0/2965355077 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbe700fc4d0 msgr2=0x7fbe700fc8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:50:04.497 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:04.493+0000 7fbe77dfe700 1 --2- 172.21.15.64:0/2965355077 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbe700fc4d0 0x7fbe700fc8f0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 rx=0x7fbe6000b5f0 tx=0x7fbe60006ee0).stop 2024-02-20T12:50:04.497 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:04.493+0000 7fbe77dfe700 1 -- 172.21.15.64:0/2965355077 shutdown_connections 2024-02-20T12:50:04.497 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:04.493+0000 7fbe77dfe700 1 --2- 172.21.15.64:0/2965355077 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbe700fc4d0 0x7fbe700fc8f0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:50:04.497 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:04.493+0000 7fbe77dfe700 1 -- 172.21.15.64:0/2965355077 >> 172.21.15.64:0/2965355077 conn(0x7fbe700f5e20 msgr2=0x7fbe700f8240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:50:04.497 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:04.493+0000 7fbe77dfe700 1 -- 172.21.15.64:0/2965355077 shutdown_connections 2024-02-20T12:50:04.497 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:04.493+0000 7fbe77dfe700 1 -- 172.21.15.64:0/2965355077 wait complete. 2024-02-20T12:50:04.497 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:04.494+0000 7fbe77dfe700 1 Processor -- start 2024-02-20T12:50:04.497 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:04.494+0000 7fbe77dfe700 1 -- start start 2024-02-20T12:50:04.497 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:04.494+0000 7fbe77dfe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbe700fc4d0 0x7fbe7010cf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:50:04.498 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:04.494+0000 7fbe77dfe700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe7010d420 con 0x7fbe700fc4d0 2024-02-20T12:50:04.498 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:04.494+0000 7fbe75b9a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbe700fc4d0 0x7fbe7010cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:50:04.498 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:04.494+0000 7fbe75b9a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbe700fc4d0 0x7fbe7010cf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:40950/0 (socket says 172.21.15.64:40950) 2024-02-20T12:50:04.498 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:04.494+0000 7fbe75b9a700 1 -- 172.21.15.64:0/2769346405 learned_addr learned my addr 172.21.15.64:0/2769346405 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-20T12:50:04.498 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:04.495+0000 7fbe75b9a700 1 -- 172.21.15.64:0/2769346405 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe6000b200 con 0x7fbe700fc4d0 2024-02-20T12:50:04.498 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:04.495+0000 7fbe75b9a700 1 --2- 172.21.15.64:0/2769346405 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbe700fc4d0 0x7fbe7010cf30 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 rx=0x7fbe60006800 tx=0x7fbe60006ee0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:50:04.498 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:04.495+0000 7fbe5e7fc700 1 -- 172.21.15.64:0/2769346405 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fbe6000de50 con 0x7fbe700fc4d0 2024-02-20T12:50:04.498 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:04.495+0000 7fbe77dfe700 1 -- 172.21.15.64:0/2769346405 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe7010d620 con 0x7fbe700fc4d0 2024-02-20T12:50:04.499 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:04.495+0000 7fbe5e7fc700 1 -- 172.21.15.64:0/2769346405 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(25 keys) v1 ==== 997+0+0 (secure 0 0 0) 0x7fbe6001b870 con 0x7fbe700fc4d0 2024-02-20T12:50:04.499 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:04.495+0000 7fbe5e7fc700 1 -- 172.21.15.64:0/2769346405 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fbe6001a7a0 con 0x7fbe700fc4d0 2024-02-20T12:50:04.499 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:04.495+0000 7fbe77dfe700 1 -- 172.21.15.64:0/2769346405 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe7010dab0 con 0x7fbe700fc4d0 2024-02-20T12:50:04.499 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:04.495+0000 7fbe5e7fc700 1 -- 172.21.15.64:0/2769346405 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 1) v1 ==== 491+0+0 (secure 0 0 0) 0x7fbe6000f070 con 0x7fbe700fc4d0 2024-02-20T12:50:04.499 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:04.495+0000 7fbe5e7fc700 1 -- 172.21.15.64:0/2769346405 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7fbe60024c50 con 0x7fbe700fc4d0 2024-02-20T12:50:04.499 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:04.496+0000 7fbe77dfe700 1 -- 172.21.15.64:0/2769346405 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe70045bb0 con 0x7fbe700fc4d0 2024-02-20T12:50:04.499 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:04.498+0000 7fbe5e7fc700 1 -- 172.21.15.64:0/2769346405 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72097 (secure 0 0 0) 0x7fbe6001f030 con 0x7fbe700fc4d0 2024-02-20T12:50:04.566 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:04.565+0000 7fbe77dfe700 1 -- 172.21.15.64:0/2769346405 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) v1 -- 0x7fbe700faaf0 con 0x7fbe700fc4d0 2024-02-20T12:50:04.568 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:04.567+0000 7fbe5e7fc700 1 -- 172.21.15.64:0/2769346405 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) v1 ==== 124+0+0 (secure 0 0 0) 0x7fbe600330c0 con 0x7fbe700fc4d0 2024-02-20T12:50:04.569 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:04.568+0000 7fbe77dfe700 1 -- 172.21.15.64:0/2769346405 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbe700fc4d0 msgr2=0x7fbe7010cf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:50:04.570 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:04.568+0000 7fbe77dfe700 1 --2- 172.21.15.64:0/2769346405 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbe700fc4d0 0x7fbe7010cf30 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 rx=0x7fbe60006800 tx=0x7fbe60006ee0).stop 2024-02-20T12:50:04.570 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:04.568+0000 7fbe77dfe700 1 -- 172.21.15.64:0/2769346405 shutdown_connections 2024-02-20T12:50:04.570 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:04.568+0000 7fbe77dfe700 1 --2- 172.21.15.64:0/2769346405 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbe700fc4d0 0x7fbe7010cf30 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:50:04.570 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:04.568+0000 7fbe77dfe700 1 -- 172.21.15.64:0/2769346405 >> 172.21.15.64:0/2769346405 conn(0x7fbe700f5e20 msgr2=0x7fbe700f8210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:50:04.570 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:04.568+0000 7fbe77dfe700 1 -- 172.21.15.64:0/2769346405 shutdown_connections 2024-02-20T12:50:04.570 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:04.568+0000 7fbe77dfe700 1 -- 172.21.15.64:0/2769346405 wait complete. 2024-02-20T12:50:05.236 INFO:teuthology.orchestra.run.smithi064.stderr:Wrote config to /etc/ceph/ceph.conf 2024-02-20T12:50:05.236 INFO:teuthology.orchestra.run.smithi064.stderr:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-02-20T12:50:05.236 INFO:teuthology.orchestra.run.smithi064.stderr:Creating mgr... 2024-02-20T12:50:05.236 INFO:teuthology.orchestra.run.smithi064.stderr:Verifying port 9283 ... 2024-02-20T12:50:05.373 INFO:teuthology.orchestra.run.smithi064.stderr:systemctl: Failed to reset failed state of unit ceph-680ee126-cfee-11ee-95bb-87774f69a715@mgr.smithi064.vriyor.service: Unit ceph-680ee126-cfee-11ee-95bb-87774f69a715@mgr.smithi064.vriyor.service not loaded. 2024-02-20T12:50:05.380 INFO:teuthology.orchestra.run.smithi064.stderr:systemctl: Created symlink /etc/systemd/system/ceph-680ee126-cfee-11ee-95bb-87774f69a715.target.wants/ceph-680ee126-cfee-11ee-95bb-87774f69a715@mgr.smithi064.vriyor.service → /etc/systemd/system/ceph-680ee126-cfee-11ee-95bb-87774f69a715@.service. 2024-02-20T12:50:05.866 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:05 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:50:04.568694+0000 mon.smithi064 (mon.0) 6 : audit [INF] from='client.? 172.21.15.64:0/2769346405' entity='client.admin' 2024-02-20T12:50:06.124 INFO:teuthology.orchestra.run.smithi064.stderr:systemctl: disabled 2024-02-20T12:50:06.133 INFO:teuthology.orchestra.run.smithi064.stderr:systemctl: inactive 2024-02-20T12:50:06.134 INFO:teuthology.orchestra.run.smithi064.stderr:firewalld.service is not enabled 2024-02-20T12:50:06.134 INFO:teuthology.orchestra.run.smithi064.stderr:Not possible to enable service . firewalld.service is not available 2024-02-20T12:50:06.142 INFO:teuthology.orchestra.run.smithi064.stderr:systemctl: disabled 2024-02-20T12:50:06.151 INFO:teuthology.orchestra.run.smithi064.stderr:systemctl: inactive 2024-02-20T12:50:06.152 INFO:teuthology.orchestra.run.smithi064.stderr:firewalld.service is not enabled 2024-02-20T12:50:06.152 INFO:teuthology.orchestra.run.smithi064.stderr:Not possible to open ports <[9283]>. firewalld.service is not available 2024-02-20T12:50:06.152 INFO:teuthology.orchestra.run.smithi064.stderr:Waiting for mgr to start... 2024-02-20T12:50:06.152 INFO:teuthology.orchestra.run.smithi064.stderr:Waiting for mgr... 2024-02-20T12:50:06.841 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:06.828+0000 7f8a11b27700 1 Processor -- start 2024-02-20T12:50:06.841 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:06.828+0000 7f8a11b27700 1 -- start start 2024-02-20T12:50:06.841 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:06.828+0000 7f8a11b27700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8a0c0fa2b0 0x7f8a0c0fa6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-02-20T12:50:06.841 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:06.828+0000 7f8a11b27700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a0c0fabc0 con 0x7f8a0c0fa2b0 2024-02-20T12:50:06.841 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:06.829+0000 7f8a0b7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8a0c0fa2b0 0x7f8a0c0fa6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:50:06.841 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:06.829+0000 7f8a0b7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8a0c0fa2b0 0x7f8a0c0fa6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:40976/0 (socket says 172.21.15.64:40976) 2024-02-20T12:50:06.841 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:06.829+0000 7f8a0b7fe700 1 -- 172.21.15.64:0/2717830151 learned_addr learned my addr 172.21.15.64:0/2717830151 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-20T12:50:06.841 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:06.829+0000 7f8a0b7fe700 1 -- 172.21.15.64:0/2717830151 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a0c0fad00 con 0x7f8a0c0fa2b0 2024-02-20T12:50:06.841 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:06.829+0000 7f8a0b7fe700 1 --2- 172.21.15.64:0/2717830151 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8a0c0fa2b0 0x7f8a0c0fa6d0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 rx=0x7f89fc00b5f0 tx=0x7f89fc006ee0).ready entity=mon.0 client_cookie=dd968a47964a4499 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:50:06.842 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:06.829+0000 7f8a0a7fc700 1 -- 172.21.15.64:0/2717830151 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f89fc00d600 con 0x7f8a0c0fa2b0 2024-02-20T12:50:06.842 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:06.829+0000 7f8a0a7fc700 1 -- 172.21.15.64:0/2717830151 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(25 keys) v1 ==== 997+0+0 (secure 0 0 0) 0x7f89fc00dbc0 con 0x7f8a0c0fa2b0 2024-02-20T12:50:06.842 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:06.829+0000 7f8a0a7fc700 1 -- 172.21.15.64:0/2717830151 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f89fc011b90 con 0x7f8a0c0fa2b0 2024-02-20T12:50:06.842 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:06.830+0000 7f8a11b27700 1 -- 172.21.15.64:0/2717830151 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8a0c0fa2b0 msgr2=0x7f8a0c0fa6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:50:06.842 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:06.830+0000 7f8a11b27700 1 --2- 172.21.15.64:0/2717830151 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8a0c0fa2b0 0x7f8a0c0fa6d0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 rx=0x7f89fc00b5f0 tx=0x7f89fc006ee0).stop 2024-02-20T12:50:06.842 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:06.830+0000 7f8a11b27700 1 -- 172.21.15.64:0/2717830151 shutdown_connections 2024-02-20T12:50:06.842 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:06.830+0000 7f8a11b27700 1 --2- 172.21.15.64:0/2717830151 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8a0c0fa2b0 0x7f8a0c0fa6d0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:50:06.842 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:06.830+0000 7f8a11b27700 1 -- 172.21.15.64:0/2717830151 >> 172.21.15.64:0/2717830151 conn(0x7f8a0c0f5df0 msgr2=0x7f8a0c0f8210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:50:06.842 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:06.830+0000 7f8a11b27700 1 -- 172.21.15.64:0/2717830151 shutdown_connections 2024-02-20T12:50:06.842 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:06.830+0000 7f8a11b27700 1 -- 172.21.15.64:0/2717830151 wait complete. 2024-02-20T12:50:06.842 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:06.830+0000 7f8a11b27700 1 Processor -- start 2024-02-20T12:50:06.842 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:06.831+0000 7f8a11b27700 1 -- start start 2024-02-20T12:50:06.842 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:06.831+0000 7f8a11b27700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8a0c0fa2b0 0x7f8a0c0fce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:50:06.842 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:06.831+0000 7f8a11b27700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a0c0fb2e0 con 0x7f8a0c0fa2b0 2024-02-20T12:50:06.842 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:06.831+0000 7f8a0b7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8a0c0fa2b0 0x7f8a0c0fce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:50:06.843 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:06.831+0000 7f8a0b7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8a0c0fa2b0 0x7f8a0c0fce60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:40978/0 (socket says 172.21.15.64:40978) 2024-02-20T12:50:06.843 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:06.831+0000 7f8a0b7fe700 1 -- 172.21.15.64:0/3099525677 learned_addr learned my addr 172.21.15.64:0/3099525677 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-20T12:50:06.843 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:06.831+0000 7f8a0b7fe700 1 -- 172.21.15.64:0/3099525677 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89fc00b200 con 0x7f8a0c0fa2b0 2024-02-20T12:50:06.843 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:06.831+0000 7f8a0b7fe700 1 --2- 172.21.15.64:0/3099525677 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8a0c0fa2b0 0x7f8a0c0fce60 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 rx=0x7f89fc00e040 tx=0x7f89fc006ee0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:50:06.843 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:06.831+0000 7f89f3fff700 1 -- 172.21.15.64:0/3099525677 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f89fc00de00 con 0x7f8a0c0fa2b0 2024-02-20T12:50:06.843 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:06.831+0000 7f8a11b27700 1 -- 172.21.15.64:0/3099525677 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a0c0fb480 con 0x7f8a0c0fa2b0 2024-02-20T12:50:06.843 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:06.831+0000 7f89f3fff700 1 -- 172.21.15.64:0/3099525677 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(25 keys) v1 ==== 997+0+0 (secure 0 0 0) 0x7f89fc01b870 con 0x7f8a0c0fa2b0 2024-02-20T12:50:06.843 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:06.831+0000 7f89f3fff700 1 -- 172.21.15.64:0/3099525677 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f89fc01a6c0 con 0x7f8a0c0fa2b0 2024-02-20T12:50:06.843 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:06.832+0000 7f8a11b27700 1 -- 172.21.15.64:0/3099525677 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a0c0fb910 con 0x7f8a0c0fa2b0 2024-02-20T12:50:06.843 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:06.832+0000 7f89f3fff700 1 -- 172.21.15.64:0/3099525677 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 1) v1 ==== 491+0+0 (secure 0 0 0) 0x7f89fc00f070 con 0x7f8a0c0fa2b0 2024-02-20T12:50:06.843 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:06.832+0000 7f89f3fff700 1 -- 172.21.15.64:0/3099525677 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f89fc024b50 con 0x7f8a0c0fa2b0 2024-02-20T12:50:06.843 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:06.832+0000 7f8a11b27700 1 -- 172.21.15.64:0/3099525677 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a0c045bb0 con 0x7f8a0c0fa2b0 2024-02-20T12:50:06.843 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:06.835+0000 7f89f3fff700 1 -- 172.21.15.64:0/3099525677 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72097 (secure 0 0 0) 0x7f89fc01f030 con 0x7f8a0c0fa2b0 2024-02-20T12:50:06.912 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:06.904+0000 7f8a11b27700 1 -- 172.21.15.64:0/3099525677 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f8a0c1047b0 con 0x7f8a0c0fa2b0 2024-02-20T12:50:06.912 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:06.905+0000 7f89f3fff700 1 -- 172.21.15.64:0/3099525677 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1249 (secure 0 0 0) 0x7f89fc032090 con 0x7f8a0c0fa2b0 2024-02-20T12:50:06.912 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:06.912 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: { 2024-02-20T12:50:06.912 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: "fsid": "680ee126-cfee-11ee-95bb-87774f69a715", 2024-02-20T12:50:06.913 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: "health": { 2024-02-20T12:50:06.913 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: "status": "HEALTH_OK", 2024-02-20T12:50:06.913 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: "checks": {}, 2024-02-20T12:50:06.913 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: "mutes": [] 2024-02-20T12:50:06.913 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: }, 2024-02-20T12:50:06.913 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: "election_epoch": 5, 2024-02-20T12:50:06.913 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: "quorum": [ 2024-02-20T12:50:06.913 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 0 2024-02-20T12:50:06.913 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: ], 2024-02-20T12:50:06.913 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: "quorum_names": [ 2024-02-20T12:50:06.913 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: "smithi064" 2024-02-20T12:50:06.913 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: ], 2024-02-20T12:50:06.913 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: "quorum_age": 3, 2024-02-20T12:50:06.913 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: "monmap": { 2024-02-20T12:50:06.913 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: "epoch": 1, 2024-02-20T12:50:06.914 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: "min_mon_release_name": "pacific", 2024-02-20T12:50:06.914 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: "num_mons": 1 2024-02-20T12:50:06.914 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: }, 2024-02-20T12:50:06.914 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: "osdmap": { 2024-02-20T12:50:06.914 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: "epoch": 1, 2024-02-20T12:50:06.914 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: "num_osds": 0, 2024-02-20T12:50:06.915 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: "num_up_osds": 0, 2024-02-20T12:50:06.915 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: "osd_up_since": 0, 2024-02-20T12:50:06.915 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: "num_in_osds": 0, 2024-02-20T12:50:06.915 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: "osd_in_since": 0, 2024-02-20T12:50:06.915 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: "num_remapped_pgs": 0 2024-02-20T12:50:06.915 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: }, 2024-02-20T12:50:06.915 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: "pgmap": { 2024-02-20T12:50:06.915 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: "pgs_by_state": [], 2024-02-20T12:50:06.915 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: "num_pgs": 0, 2024-02-20T12:50:06.915 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: "num_pools": 0, 2024-02-20T12:50:06.915 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: "num_objects": 0, 2024-02-20T12:50:06.915 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: "data_bytes": 0, 2024-02-20T12:50:06.915 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: "bytes_used": 0, 2024-02-20T12:50:06.915 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: "bytes_avail": 0, 2024-02-20T12:50:06.915 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: "bytes_total": 0 2024-02-20T12:50:06.916 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: }, 2024-02-20T12:50:06.916 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: "fsmap": { 2024-02-20T12:50:06.916 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: "epoch": 1, 2024-02-20T12:50:06.916 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: "by_rank": [], 2024-02-20T12:50:06.916 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: "up:standby": 0 2024-02-20T12:50:06.916 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: }, 2024-02-20T12:50:06.916 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: "mgrmap": { 2024-02-20T12:50:06.916 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: "available": false, 2024-02-20T12:50:06.916 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: "num_standbys": 0, 2024-02-20T12:50:06.916 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: "modules": [ 2024-02-20T12:50:06.916 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: "iostat", 2024-02-20T12:50:06.916 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: "nfs", 2024-02-20T12:50:06.916 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: "restful" 2024-02-20T12:50:06.916 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: ], 2024-02-20T12:50:06.916 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: "services": {} 2024-02-20T12:50:06.917 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: }, 2024-02-20T12:50:06.917 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: "servicemap": { 2024-02-20T12:50:06.917 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: "epoch": 1, 2024-02-20T12:50:06.917 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: "modified": "2024-02-20T12:49:48.181169+0000", 2024-02-20T12:50:06.917 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: "services": {} 2024-02-20T12:50:06.917 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: }, 2024-02-20T12:50:06.917 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: "progress_events": {} 2024-02-20T12:50:06.917 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: } 2024-02-20T12:50:06.917 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:06.906+0000 7f8a11b27700 1 -- 172.21.15.64:0/3099525677 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8a0c0fa2b0 msgr2=0x7f8a0c0fce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:50:06.917 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:06.906+0000 7f8a11b27700 1 --2- 172.21.15.64:0/3099525677 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8a0c0fa2b0 0x7f8a0c0fce60 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 rx=0x7f89fc00e040 tx=0x7f89fc006ee0).stop 2024-02-20T12:50:06.917 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:06.906+0000 7f8a11b27700 1 -- 172.21.15.64:0/3099525677 shutdown_connections 2024-02-20T12:50:06.917 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:06.906+0000 7f8a11b27700 1 --2- 172.21.15.64:0/3099525677 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8a0c0fa2b0 0x7f8a0c0fce60 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:50:06.917 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:06.906+0000 7f8a11b27700 1 -- 172.21.15.64:0/3099525677 >> 172.21.15.64:0/3099525677 conn(0x7f8a0c0f5df0 msgr2=0x7f8a0c0ff4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:50:06.917 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:06.906+0000 7f8a11b27700 1 -- 172.21.15.64:0/3099525677 shutdown_connections 2024-02-20T12:50:06.917 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:06.906+0000 7f8a11b27700 1 -- 172.21.15.64:0/3099525677 wait complete. 2024-02-20T12:50:07.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:06 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:50:06.905870+0000 mon.smithi064 (mon.0) 7 : audit [DBG] from='client.? 172.21.15.64:0/3099525677' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-02-20T12:50:07.699 INFO:teuthology.orchestra.run.smithi064.stderr:mgr not available, waiting (1/15)... 2024-02-20T12:50:10.334 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:10.280+0000 7f76fdde1700 1 Processor -- start 2024-02-20T12:50:10.334 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:10.280+0000 7f76fdde1700 1 -- start start 2024-02-20T12:50:10.334 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:10.280+0000 7f76fdde1700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f76f80fad70 0x7f76f80fb190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-02-20T12:50:10.334 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:10.280+0000 7f76fdde1700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76f80fb680 con 0x7f76f80fad70 2024-02-20T12:50:10.334 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:10.280+0000 7f76fcddf700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f76f80fad70 0x7f76f80fb190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:50:10.334 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:10.280+0000 7f76fcddf700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f76f80fad70 0x7f76f80fb190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:40988/0 (socket says 172.21.15.64:40988) 2024-02-20T12:50:10.334 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:10.280+0000 7f76fcddf700 1 -- 172.21.15.64:0/2426884090 learned_addr learned my addr 172.21.15.64:0/2426884090 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-20T12:50:10.334 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:10.281+0000 7f76fcddf700 1 -- 172.21.15.64:0/2426884090 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76f80fb7c0 con 0x7f76f80fad70 2024-02-20T12:50:10.334 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:10.281+0000 7f76fcddf700 1 --2- 172.21.15.64:0/2426884090 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f76f80fad70 0x7f76f80fb190 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0x7f76ec00b5f0 tx=0x7f76ec006ee0).ready entity=mon.0 client_cookie=88eef1775aa40966 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:50:10.335 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:10.281+0000 7f76f77fe700 1 -- 172.21.15.64:0/2426884090 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f76ec00d600 con 0x7f76f80fad70 2024-02-20T12:50:10.335 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:10.281+0000 7f76f77fe700 1 -- 172.21.15.64:0/2426884090 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(25 keys) v1 ==== 997+0+0 (secure 0 0 0) 0x7f76ec00dbc0 con 0x7f76f80fad70 2024-02-20T12:50:10.335 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:10.281+0000 7f76f77fe700 1 -- 172.21.15.64:0/2426884090 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f76ec011b90 con 0x7f76f80fad70 2024-02-20T12:50:10.335 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:10.281+0000 7f76fdde1700 1 -- 172.21.15.64:0/2426884090 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f76f80fad70 msgr2=0x7f76f80fb190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:50:10.335 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:10.281+0000 7f76fdde1700 1 --2- 172.21.15.64:0/2426884090 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f76f80fad70 0x7f76f80fb190 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0x7f76ec00b5f0 tx=0x7f76ec006ee0).stop 2024-02-20T12:50:10.335 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:10.281+0000 7f76fdde1700 1 -- 172.21.15.64:0/2426884090 shutdown_connections 2024-02-20T12:50:10.335 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:10.281+0000 7f76fdde1700 1 --2- 172.21.15.64:0/2426884090 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f76f80fad70 0x7f76f80fb190 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:50:10.335 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:10.281+0000 7f76fdde1700 1 -- 172.21.15.64:0/2426884090 >> 172.21.15.64:0/2426884090 conn(0x7f76f80f64d0 msgr2=0x7f76f80f88f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:50:10.335 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:10.282+0000 7f76fdde1700 1 -- 172.21.15.64:0/2426884090 shutdown_connections 2024-02-20T12:50:10.335 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:10.282+0000 7f76fdde1700 1 -- 172.21.15.64:0/2426884090 wait complete. 2024-02-20T12:50:10.335 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:10.282+0000 7f76fdde1700 1 Processor -- start 2024-02-20T12:50:10.336 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:10.282+0000 7f76fdde1700 1 -- start start 2024-02-20T12:50:10.336 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:10.282+0000 7f76fdde1700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f76f80fad70 0x7f76f810cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:50:10.336 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:10.282+0000 7f76fdde1700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76f810d260 con 0x7f76f80fad70 2024-02-20T12:50:10.336 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:10.283+0000 7f76fcddf700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f76f80fad70 0x7f76f810cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:50:10.336 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:10.283+0000 7f76fcddf700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f76f80fad70 0x7f76f810cd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:41004/0 (socket says 172.21.15.64:41004) 2024-02-20T12:50:10.336 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:10.283+0000 7f76fcddf700 1 -- 172.21.15.64:0/2621686880 learned_addr learned my addr 172.21.15.64:0/2621686880 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-20T12:50:10.336 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:10.283+0000 7f76fcddf700 1 -- 172.21.15.64:0/2621686880 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76ec00b200 con 0x7f76f80fad70 2024-02-20T12:50:10.336 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:10.283+0000 7f76fcddf700 1 --2- 172.21.15.64:0/2621686880 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f76f80fad70 0x7f76f810cd70 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0x7f76ec00e040 tx=0x7f76ec006ee0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:50:10.336 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:10.283+0000 7f76f57fa700 1 -- 172.21.15.64:0/2621686880 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f76ec00de00 con 0x7f76f80fad70 2024-02-20T12:50:10.336 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:10.283+0000 7f76f57fa700 1 -- 172.21.15.64:0/2621686880 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(25 keys) v1 ==== 997+0+0 (secure 0 0 0) 0x7f76ec01b870 con 0x7f76f80fad70 2024-02-20T12:50:10.337 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:10.283+0000 7f76fdde1700 1 -- 172.21.15.64:0/2621686880 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76f810d460 con 0x7f76f80fad70 2024-02-20T12:50:10.337 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:10.283+0000 7f76f57fa700 1 -- 172.21.15.64:0/2621686880 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f76ec01a6c0 con 0x7f76f80fad70 2024-02-20T12:50:10.337 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:10.283+0000 7f76fdde1700 1 -- 172.21.15.64:0/2621686880 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76f810e070 con 0x7f76f80fad70 2024-02-20T12:50:10.337 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:10.284+0000 7f76f57fa700 1 -- 172.21.15.64:0/2621686880 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 1) v1 ==== 491+0+0 (secure 0 0 0) 0x7f76ec00f070 con 0x7f76f80fad70 2024-02-20T12:50:10.337 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:10.284+0000 7f76fdde1700 1 -- 172.21.15.64:0/2621686880 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76f80591e0 con 0x7f76f80fad70 2024-02-20T12:50:10.337 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:10.284+0000 7f76f57fa700 1 -- 172.21.15.64:0/2621686880 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f76ec024760 con 0x7f76f80fad70 2024-02-20T12:50:10.337 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:10.286+0000 7f76f57fa700 1 -- 172.21.15.64:0/2621686880 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72097 (secure 0 0 0) 0x7f76ec01f030 con 0x7f76f80fad70 2024-02-20T12:50:10.366 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:10.357+0000 7f76fdde1700 1 -- 172.21.15.64:0/2621686880 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f76f80fcba0 con 0x7f76f80fad70 2024-02-20T12:50:10.366 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:10.357+0000 7f76f57fa700 1 -- 172.21.15.64:0/2621686880 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1249 (secure 0 0 0) 0x7f76ec032090 con 0x7f76f80fad70 2024-02-20T12:50:10.367 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:10.367 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: { 2024-02-20T12:50:10.367 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: "fsid": "680ee126-cfee-11ee-95bb-87774f69a715", 2024-02-20T12:50:10.367 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: "health": { 2024-02-20T12:50:10.367 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: "status": "HEALTH_OK", 2024-02-20T12:50:10.367 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: "checks": {}, 2024-02-20T12:50:10.367 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: "mutes": [] 2024-02-20T12:50:10.367 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: }, 2024-02-20T12:50:10.367 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: "election_epoch": 5, 2024-02-20T12:50:10.367 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: "quorum": [ 2024-02-20T12:50:10.367 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 0 2024-02-20T12:50:10.367 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: ], 2024-02-20T12:50:10.367 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: "quorum_names": [ 2024-02-20T12:50:10.367 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: "smithi064" 2024-02-20T12:50:10.368 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: ], 2024-02-20T12:50:10.368 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: "quorum_age": 6, 2024-02-20T12:50:10.368 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: "monmap": { 2024-02-20T12:50:10.368 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: "epoch": 1, 2024-02-20T12:50:10.368 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: "min_mon_release_name": "pacific", 2024-02-20T12:50:10.368 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: "num_mons": 1 2024-02-20T12:50:10.368 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: }, 2024-02-20T12:50:10.368 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: "osdmap": { 2024-02-20T12:50:10.368 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: "epoch": 1, 2024-02-20T12:50:10.368 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: "num_osds": 0, 2024-02-20T12:50:10.368 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: "num_up_osds": 0, 2024-02-20T12:50:10.368 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: "osd_up_since": 0, 2024-02-20T12:50:10.369 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: "num_in_osds": 0, 2024-02-20T12:50:10.369 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: "osd_in_since": 0, 2024-02-20T12:50:10.369 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: "num_remapped_pgs": 0 2024-02-20T12:50:10.369 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: }, 2024-02-20T12:50:10.369 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: "pgmap": { 2024-02-20T12:50:10.369 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: "pgs_by_state": [], 2024-02-20T12:50:10.369 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: "num_pgs": 0, 2024-02-20T12:50:10.369 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: "num_pools": 0, 2024-02-20T12:50:10.369 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: "num_objects": 0, 2024-02-20T12:50:10.370 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: "data_bytes": 0, 2024-02-20T12:50:10.370 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: "bytes_used": 0, 2024-02-20T12:50:10.370 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: "bytes_avail": 0, 2024-02-20T12:50:10.370 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: "bytes_total": 0 2024-02-20T12:50:10.370 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: }, 2024-02-20T12:50:10.370 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: "fsmap": { 2024-02-20T12:50:10.370 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: "epoch": 1, 2024-02-20T12:50:10.370 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: "by_rank": [], 2024-02-20T12:50:10.370 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: "up:standby": 0 2024-02-20T12:50:10.370 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: }, 2024-02-20T12:50:10.371 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: "mgrmap": { 2024-02-20T12:50:10.371 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: "available": false, 2024-02-20T12:50:10.371 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: "num_standbys": 0, 2024-02-20T12:50:10.371 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: "modules": [ 2024-02-20T12:50:10.371 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: "iostat", 2024-02-20T12:50:10.371 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: "nfs", 2024-02-20T12:50:10.371 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: "restful" 2024-02-20T12:50:10.371 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: ], 2024-02-20T12:50:10.371 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: "services": {} 2024-02-20T12:50:10.372 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: }, 2024-02-20T12:50:10.372 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: "servicemap": { 2024-02-20T12:50:10.372 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: "epoch": 1, 2024-02-20T12:50:10.372 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: "modified": "2024-02-20T12:49:48.181169+0000", 2024-02-20T12:50:10.372 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: "services": {} 2024-02-20T12:50:10.372 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: }, 2024-02-20T12:50:10.372 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: "progress_events": {} 2024-02-20T12:50:10.372 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: } 2024-02-20T12:50:10.373 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:10.358+0000 7f76fdde1700 1 -- 172.21.15.64:0/2621686880 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f76f80fad70 msgr2=0x7f76f810cd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:50:10.373 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:10.358+0000 7f76fdde1700 1 --2- 172.21.15.64:0/2621686880 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f76f80fad70 0x7f76f810cd70 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0x7f76ec00e040 tx=0x7f76ec006ee0).stop 2024-02-20T12:50:10.373 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:10.358+0000 7f76fdde1700 1 -- 172.21.15.64:0/2621686880 shutdown_connections 2024-02-20T12:50:10.373 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:10.358+0000 7f76fdde1700 1 --2- 172.21.15.64:0/2621686880 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f76f80fad70 0x7f76f810cd70 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:50:10.373 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:10.358+0000 7f76fdde1700 1 -- 172.21.15.64:0/2621686880 >> 172.21.15.64:0/2621686880 conn(0x7f76f80f64d0 msgr2=0x7f76f80f70d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:50:10.373 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:10.359+0000 7f76fdde1700 1 -- 172.21.15.64:0/2621686880 shutdown_connections 2024-02-20T12:50:10.373 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:10.359+0000 7f76fdde1700 1 -- 172.21.15.64:0/2621686880 wait complete. 2024-02-20T12:50:10.782 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:10 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:50:10.358735+0000 mon.smithi064 (mon.0) 8 : audit [DBG] from='client.? 172.21.15.64:0/2621686880' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-02-20T12:50:11.118 INFO:teuthology.orchestra.run.smithi064.stderr:mgr not available, waiting (2/15)... 2024-02-20T12:50:11.812 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:11 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:50:11.378896+0000 mon.smithi064 (mon.0) 9 : cluster [INF] Activating manager daemon smithi064.vriyor 2024-02-20T12:50:11.813 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:11 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:50:11.381517+0000 mon.smithi064 (mon.0) 10 : cluster [DBG] mgrmap e2: smithi064.vriyor(active, starting, since 0.00272461s) 2024-02-20T12:50:11.813 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:11 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:50:11.382487+0000 mon.smithi064 (mon.0) 11 : audit [DBG] from='mgr.14100 172.21.15.64:0/3952064285' entity='mgr.smithi064.vriyor' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-20T12:50:11.813 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:11 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:50:11.382620+0000 mon.smithi064 (mon.0) 12 : audit [DBG] from='mgr.14100 172.21.15.64:0/3952064285' entity='mgr.smithi064.vriyor' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-20T12:50:11.813 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:11 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:50:11.382732+0000 mon.smithi064 (mon.0) 13 : audit [DBG] from='mgr.14100 172.21.15.64:0/3952064285' entity='mgr.smithi064.vriyor' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-20T12:50:11.813 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:11 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:50:11.382885+0000 mon.smithi064 (mon.0) 14 : audit [DBG] from='mgr.14100 172.21.15.64:0/3952064285' entity='mgr.smithi064.vriyor' cmd=[{"prefix": "mon metadata", "id": "smithi064"}]: dispatch 2024-02-20T12:50:11.813 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:11 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:50:11.383206+0000 mon.smithi064 (mon.0) 15 : audit [DBG] from='mgr.14100 172.21.15.64:0/3952064285' entity='mgr.smithi064.vriyor' cmd=[{"prefix": "mgr metadata", "who": "smithi064.vriyor", "id": "smithi064.vriyor"}]: dispatch 2024-02-20T12:50:11.813 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:11 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:50:11.397269+0000 mon.smithi064 (mon.0) 16 : cluster [INF] Manager daemon smithi064.vriyor is now available 2024-02-20T12:50:11.813 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:11 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:50:11.405182+0000 mon.smithi064 (mon.0) 17 : audit [INF] from='mgr.14100 172.21.15.64:0/3952064285' entity='mgr.smithi064.vriyor' 2024-02-20T12:50:12.812 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:12 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:50:11.411661+0000 mon.smithi064 (mon.0) 18 : audit [INF] from='mgr.14100 172.21.15.64:0/3952064285' entity='mgr.smithi064.vriyor' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi064.vriyor/mirror_snapshot_schedule"}]: dispatch 2024-02-20T12:50:12.812 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:12 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:50:11.413044+0000 mon.smithi064 (mon.0) 19 : audit [INF] from='mgr.14100 172.21.15.64:0/3952064285' entity='mgr.smithi064.vriyor' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi064.vriyor/trash_purge_schedule"}]: dispatch 2024-02-20T12:50:12.812 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:12 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:50:11.418305+0000 mon.smithi064 (mon.0) 20 : audit [INF] from='mgr.14100 172.21.15.64:0/3952064285' entity='mgr.smithi064.vriyor' 2024-02-20T12:50:12.812 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:12 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:50:11.420256+0000 mon.smithi064 (mon.0) 21 : audit [INF] from='mgr.14100 172.21.15.64:0/3952064285' entity='mgr.smithi064.vriyor' 2024-02-20T12:50:12.813 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:12 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:50:12.384953+0000 mon.smithi064 (mon.0) 22 : cluster [DBG] mgrmap e3: smithi064.vriyor(active, since 1.00616s) 2024-02-20T12:50:13.885 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:13.880+0000 7fc9b13a8700 1 Processor -- start 2024-02-20T12:50:13.885 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:13.880+0000 7fc9b13a8700 1 -- start start 2024-02-20T12:50:13.885 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:13.880+0000 7fc9b13a8700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc9ac0fc4d0 0x7fc9ac0fc8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-02-20T12:50:13.885 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:13.880+0000 7fc9b13a8700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9ac0fcde0 con 0x7fc9ac0fc4d0 2024-02-20T12:50:13.885 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:13.880+0000 7fc9aaffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc9ac0fc4d0 0x7fc9ac0fc8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:50:13.885 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:13.880+0000 7fc9aaffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc9ac0fc4d0 0x7fc9ac0fc8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53628/0 (socket says 172.21.15.64:53628) 2024-02-20T12:50:13.885 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:13.880+0000 7fc9aaffd700 1 -- 172.21.15.64:0/208099255 learned_addr learned my addr 172.21.15.64:0/208099255 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-20T12:50:13.885 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:13.880+0000 7fc9aaffd700 1 -- 172.21.15.64:0/208099255 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9ac0fcf20 con 0x7fc9ac0fc4d0 2024-02-20T12:50:13.885 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:13.881+0000 7fc9aaffd700 1 --2- 172.21.15.64:0/208099255 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc9ac0fc4d0 0x7fc9ac0fc8f0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 rx=0x7fc99c00b5f0 tx=0x7fc99c006ee0).ready entity=mon.0 client_cookie=3ad267f72c0673ae server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:50:13.886 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:13.881+0000 7fc9a9ffb700 1 -- 172.21.15.64:0/208099255 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc99c00d600 con 0x7fc9ac0fc4d0 2024-02-20T12:50:13.886 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:13.881+0000 7fc9a9ffb700 1 -- 172.21.15.64:0/208099255 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(25 keys) v1 ==== 997+0+0 (secure 0 0 0) 0x7fc99c00dbc0 con 0x7fc9ac0fc4d0 2024-02-20T12:50:13.886 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:13.881+0000 7fc9a9ffb700 1 -- 172.21.15.64:0/208099255 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc99c011b90 con 0x7fc9ac0fc4d0 2024-02-20T12:50:13.886 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:13.882+0000 7fc9b13a8700 1 -- 172.21.15.64:0/208099255 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc9ac0fc4d0 msgr2=0x7fc9ac0fc8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:50:13.886 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:13.882+0000 7fc9b13a8700 1 --2- 172.21.15.64:0/208099255 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc9ac0fc4d0 0x7fc9ac0fc8f0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 rx=0x7fc99c00b5f0 tx=0x7fc99c006ee0).stop 2024-02-20T12:50:13.886 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:13.882+0000 7fc9b13a8700 1 -- 172.21.15.64:0/208099255 shutdown_connections 2024-02-20T12:50:13.886 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:13.882+0000 7fc9b13a8700 1 --2- 172.21.15.64:0/208099255 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc9ac0fc4d0 0x7fc9ac0fc8f0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:50:13.886 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:13.882+0000 7fc9b13a8700 1 -- 172.21.15.64:0/208099255 >> 172.21.15.64:0/208099255 conn(0x7fc9ac0f5d80 msgr2=0x7fc9ac0f81a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:50:13.886 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:13.882+0000 7fc9b13a8700 1 -- 172.21.15.64:0/208099255 shutdown_connections 2024-02-20T12:50:13.886 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:13.882+0000 7fc9b13a8700 1 -- 172.21.15.64:0/208099255 wait complete. 2024-02-20T12:50:13.887 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:13.882+0000 7fc9b13a8700 1 Processor -- start 2024-02-20T12:50:13.887 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:13.883+0000 7fc9b13a8700 1 -- start start 2024-02-20T12:50:13.887 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:13.883+0000 7fc9b13a8700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc9ac0fc4d0 0x7fc9ac10cea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:50:13.887 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:13.883+0000 7fc9b13a8700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9ac10d390 con 0x7fc9ac0fc4d0 2024-02-20T12:50:13.887 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:13.883+0000 7fc9aaffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc9ac0fc4d0 0x7fc9ac10cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:50:13.887 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:13.883+0000 7fc9aaffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc9ac0fc4d0 0x7fc9ac10cea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53636/0 (socket says 172.21.15.64:53636) 2024-02-20T12:50:13.887 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:13.883+0000 7fc9aaffd700 1 -- 172.21.15.64:0/1766033610 learned_addr learned my addr 172.21.15.64:0/1766033610 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-20T12:50:13.889 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:13.883+0000 7fc9aaffd700 1 -- 172.21.15.64:0/1766033610 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc99c00b200 con 0x7fc9ac0fc4d0 2024-02-20T12:50:13.889 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:13.883+0000 7fc9aaffd700 1 --2- 172.21.15.64:0/1766033610 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc9ac0fc4d0 0x7fc9ac10cea0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 rx=0x7fc99c000c00 tx=0x7fc99c006ee0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:50:13.889 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:13.883+0000 7fc9937fe700 1 -- 172.21.15.64:0/1766033610 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc99c00de00 con 0x7fc9ac0fc4d0 2024-02-20T12:50:13.889 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:13.884+0000 7fc9937fe700 1 -- 172.21.15.64:0/1766033610 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(25 keys) v1 ==== 997+0+0 (secure 0 0 0) 0x7fc99c01b870 con 0x7fc9ac0fc4d0 2024-02-20T12:50:13.889 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:13.884+0000 7fc9b13a8700 1 -- 172.21.15.64:0/1766033610 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9ac10d590 con 0x7fc9ac0fc4d0 2024-02-20T12:50:13.889 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:13.884+0000 7fc9937fe700 1 -- 172.21.15.64:0/1766033610 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc99c01a670 con 0x7fc9ac0fc4d0 2024-02-20T12:50:13.889 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:13.884+0000 7fc9b13a8700 1 -- 172.21.15.64:0/1766033610 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9ac10da20 con 0x7fc9ac0fc4d0 2024-02-20T12:50:13.889 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:13.885+0000 7fc9b13a8700 1 -- 172.21.15.64:0/1766033610 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9ac045bb0 con 0x7fc9ac0fc4d0 2024-02-20T12:50:13.889 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:13.885+0000 7fc9937fe700 1 -- 172.21.15.64:0/1766033610 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 4) v1 ==== 37084+0+0 (secure 0 0 0) 0x7fc99c02b070 con 0x7fc9ac0fc4d0 2024-02-20T12:50:13.889 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:13.885+0000 7fc9937fe700 1 --2- 172.21.15.64:0/1766033610 >> [v2:172.21.15.64:6800/3845912330,v1:172.21.15.64:6801/3845912330] conn(0x7fc998030f40 0x7fc998033400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:50:13.889 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:13.885+0000 7fc9937fe700 1 -- 172.21.15.64:0/1766033610 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7fc99c042b40 con 0x7fc9ac0fc4d0 2024-02-20T12:50:13.889 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:13.885+0000 7fc9aa7fc700 1 --2- 172.21.15.64:0/1766033610 >> [v2:172.21.15.64:6800/3845912330,v1:172.21.15.64:6801/3845912330] conn(0x7fc998030f40 0x7fc998033400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:50:13.890 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:13.885+0000 7fc9aa7fc700 1 --2- 172.21.15.64:0/1766033610 >> [v2:172.21.15.64:6800/3845912330,v1:172.21.15.64:6801/3845912330] conn(0x7fc998030f40 0x7fc998033400 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 rx=0x7fc9a00096a0 tx=0x7fc9a0006b40).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:50:13.890 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:13.889+0000 7fc9937fe700 1 -- 172.21.15.64:0/1766033610 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+153237 (secure 0 0 0) 0x7fc99c01b9e0 con 0x7fc9ac0fc4d0 2024-02-20T12:50:14.114 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:14.110+0000 7fc9b13a8700 1 -- 172.21.15.64:0/1766033610 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7fc9ac102090 con 0x7fc9ac0fc4d0 2024-02-20T12:50:14.114 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:14.111+0000 7fc9937fe700 1 -- 172.21.15.64:0/1766033610 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1249 (secure 0 0 0) 0x7fc99c01f070 con 0x7fc9ac0fc4d0 2024-02-20T12:50:14.114 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:14.114 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: { 2024-02-20T12:50:14.114 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: "fsid": "680ee126-cfee-11ee-95bb-87774f69a715", 2024-02-20T12:50:14.114 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: "health": { 2024-02-20T12:50:14.115 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: "status": "HEALTH_OK", 2024-02-20T12:50:14.115 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: "checks": {}, 2024-02-20T12:50:14.115 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: "mutes": [] 2024-02-20T12:50:14.115 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: }, 2024-02-20T12:50:14.115 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: "election_epoch": 5, 2024-02-20T12:50:14.115 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: "quorum": [ 2024-02-20T12:50:14.115 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 0 2024-02-20T12:50:14.115 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: ], 2024-02-20T12:50:14.115 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: "quorum_names": [ 2024-02-20T12:50:14.115 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: "smithi064" 2024-02-20T12:50:14.115 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: ], 2024-02-20T12:50:14.115 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: "quorum_age": 10, 2024-02-20T12:50:14.115 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: "monmap": { 2024-02-20T12:50:14.115 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: "epoch": 1, 2024-02-20T12:50:14.115 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: "min_mon_release_name": "pacific", 2024-02-20T12:50:14.116 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: "num_mons": 1 2024-02-20T12:50:14.116 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: }, 2024-02-20T12:50:14.116 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: "osdmap": { 2024-02-20T12:50:14.116 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: "epoch": 1, 2024-02-20T12:50:14.116 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: "num_osds": 0, 2024-02-20T12:50:14.120 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: "num_up_osds": 0, 2024-02-20T12:50:14.121 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: "osd_up_since": 0, 2024-02-20T12:50:14.121 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: "num_in_osds": 0, 2024-02-20T12:50:14.121 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: "osd_in_since": 0, 2024-02-20T12:50:14.121 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: "num_remapped_pgs": 0 2024-02-20T12:50:14.121 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: }, 2024-02-20T12:50:14.121 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: "pgmap": { 2024-02-20T12:50:14.121 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: "pgs_by_state": [], 2024-02-20T12:50:14.121 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: "num_pgs": 0, 2024-02-20T12:50:14.121 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: "num_pools": 0, 2024-02-20T12:50:14.121 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: "num_objects": 0, 2024-02-20T12:50:14.121 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: "data_bytes": 0, 2024-02-20T12:50:14.121 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: "bytes_used": 0, 2024-02-20T12:50:14.121 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: "bytes_avail": 0, 2024-02-20T12:50:14.121 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: "bytes_total": 0 2024-02-20T12:50:14.121 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: }, 2024-02-20T12:50:14.122 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: "fsmap": { 2024-02-20T12:50:14.122 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: "epoch": 1, 2024-02-20T12:50:14.122 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: "by_rank": [], 2024-02-20T12:50:14.122 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: "up:standby": 0 2024-02-20T12:50:14.122 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: }, 2024-02-20T12:50:14.122 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: "mgrmap": { 2024-02-20T12:50:14.122 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: "available": true, 2024-02-20T12:50:14.122 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: "num_standbys": 0, 2024-02-20T12:50:14.122 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: "modules": [ 2024-02-20T12:50:14.122 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: "iostat", 2024-02-20T12:50:14.122 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: "nfs", 2024-02-20T12:50:14.123 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: "restful" 2024-02-20T12:50:14.123 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: ], 2024-02-20T12:50:14.123 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: "services": {} 2024-02-20T12:50:14.123 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: }, 2024-02-20T12:50:14.123 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: "servicemap": { 2024-02-20T12:50:14.123 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: "epoch": 1, 2024-02-20T12:50:14.123 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: "modified": "2024-02-20T12:49:48.181169+0000", 2024-02-20T12:50:14.123 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: "services": {} 2024-02-20T12:50:14.123 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: }, 2024-02-20T12:50:14.123 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: "progress_events": {} 2024-02-20T12:50:14.123 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: } 2024-02-20T12:50:14.123 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:14.112+0000 7fc9b13a8700 1 -- 172.21.15.64:0/1766033610 >> [v2:172.21.15.64:6800/3845912330,v1:172.21.15.64:6801/3845912330] conn(0x7fc998030f40 msgr2=0x7fc998033400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:50:14.123 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:14.112+0000 7fc9b13a8700 1 --2- 172.21.15.64:0/1766033610 >> [v2:172.21.15.64:6800/3845912330,v1:172.21.15.64:6801/3845912330] conn(0x7fc998030f40 0x7fc998033400 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 rx=0x7fc9a00096a0 tx=0x7fc9a0006b40).stop 2024-02-20T12:50:14.124 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:14.112+0000 7fc9b13a8700 1 -- 172.21.15.64:0/1766033610 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc9ac0fc4d0 msgr2=0x7fc9ac10cea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:50:14.124 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:14.112+0000 7fc9b13a8700 1 --2- 172.21.15.64:0/1766033610 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc9ac0fc4d0 0x7fc9ac10cea0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 rx=0x7fc99c000c00 tx=0x7fc99c006ee0).stop 2024-02-20T12:50:14.124 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:14.112+0000 7fc9b13a8700 1 -- 172.21.15.64:0/1766033610 shutdown_connections 2024-02-20T12:50:14.124 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:14.112+0000 7fc9b13a8700 1 --2- 172.21.15.64:0/1766033610 >> [v2:172.21.15.64:6800/3845912330,v1:172.21.15.64:6801/3845912330] conn(0x7fc998030f40 0x7fc998033400 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:50:14.124 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:14.112+0000 7fc9b13a8700 1 --2- 172.21.15.64:0/1766033610 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc9ac0fc4d0 0x7fc9ac10cea0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:50:14.124 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:14.112+0000 7fc9b13a8700 1 -- 172.21.15.64:0/1766033610 >> 172.21.15.64:0/1766033610 conn(0x7fc9ac0f5d80 msgr2=0x7fc9ac0f7820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:50:14.124 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:14.112+0000 7fc9b13a8700 1 -- 172.21.15.64:0/1766033610 shutdown_connections 2024-02-20T12:50:14.124 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:14.112+0000 7fc9b13a8700 1 -- 172.21.15.64:0/1766033610 wait complete. 2024-02-20T12:50:14.812 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:14 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:50:13.416857+0000 mon.smithi064 (mon.0) 23 : cluster [DBG] mgrmap e4: smithi064.vriyor(active, since 2s) 2024-02-20T12:50:14.812 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:14 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:50:14.111908+0000 mon.smithi064 (mon.0) 24 : audit [DBG] from='client.? 172.21.15.64:0/1766033610' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-02-20T12:50:14.817 INFO:teuthology.orchestra.run.smithi064.stderr:mgr is available 2024-02-20T12:50:15.366 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:15.360+0000 7feceb59e700 1 Processor -- start 2024-02-20T12:50:15.367 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:15.360+0000 7feceb59e700 1 -- start start 2024-02-20T12:50:15.367 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:15.360+0000 7feceb59e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fecec06ade0 0x7fecec0fea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-02-20T12:50:15.367 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:15.360+0000 7feceb59e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecec0fef20 con 0x7fecec06ade0 2024-02-20T12:50:15.367 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:15.360+0000 7fecea59c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fecec06ade0 0x7fecec0fea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:50:15.367 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:15.361+0000 7fecea59c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fecec06ade0 0x7fecec0fea30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53652/0 (socket says 172.21.15.64:53652) 2024-02-20T12:50:15.367 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:15.361+0000 7fecea59c700 1 -- 172.21.15.64:0/1945171263 learned_addr learned my addr 172.21.15.64:0/1945171263 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-20T12:50:15.367 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:15.361+0000 7fecea59c700 1 -- 172.21.15.64:0/1945171263 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecec0ff060 con 0x7fecec06ade0 2024-02-20T12:50:15.367 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:15.361+0000 7fecea59c700 1 --2- 172.21.15.64:0/1945171263 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fecec06ade0 0x7fecec0fea30 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 rx=0x7fecdc00b5f0 tx=0x7fecdc006ee0).ready entity=mon.0 client_cookie=34441cc0f667cbb3 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:50:15.367 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:15.361+0000 7fece959a700 1 -- 172.21.15.64:0/1945171263 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fecdc00d600 con 0x7fecec06ade0 2024-02-20T12:50:15.367 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:15.361+0000 7fece959a700 1 -- 172.21.15.64:0/1945171263 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(25 keys) v1 ==== 997+0+0 (secure 0 0 0) 0x7fecdc00dbc0 con 0x7fecec06ade0 2024-02-20T12:50:15.368 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:15.361+0000 7fece959a700 1 -- 172.21.15.64:0/1945171263 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fecdc011b90 con 0x7fecec06ade0 2024-02-20T12:50:15.368 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:15.362+0000 7feceb59e700 1 -- 172.21.15.64:0/1945171263 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fecec06ade0 msgr2=0x7fecec0fea30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:50:15.368 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:15.362+0000 7feceb59e700 1 --2- 172.21.15.64:0/1945171263 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fecec06ade0 0x7fecec0fea30 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 rx=0x7fecdc00b5f0 tx=0x7fecdc006ee0).stop 2024-02-20T12:50:15.368 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:15.362+0000 7feceb59e700 1 -- 172.21.15.64:0/1945171263 shutdown_connections 2024-02-20T12:50:15.368 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:15.362+0000 7feceb59e700 1 --2- 172.21.15.64:0/1945171263 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fecec06ade0 0x7fecec0fea30 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:50:15.368 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:15.362+0000 7feceb59e700 1 -- 172.21.15.64:0/1945171263 >> 172.21.15.64:0/1945171263 conn(0x7fecec0f64d0 msgr2=0x7fecec0f88f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:50:15.368 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:15.362+0000 7feceb59e700 1 -- 172.21.15.64:0/1945171263 shutdown_connections 2024-02-20T12:50:15.368 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:15.362+0000 7feceb59e700 1 -- 172.21.15.64:0/1945171263 wait complete. 2024-02-20T12:50:15.368 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:15.362+0000 7feceb59e700 1 Processor -- start 2024-02-20T12:50:15.368 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:15.362+0000 7feceb59e700 1 -- start start 2024-02-20T12:50:15.368 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:15.363+0000 7feceb59e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fecec06ade0 0x7fecec0fc4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:50:15.368 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:15.363+0000 7feceb59e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecec0fa930 con 0x7fecec06ade0 2024-02-20T12:50:15.368 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:15.363+0000 7fecea59c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fecec06ade0 0x7fecec0fc4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:50:15.368 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:15.363+0000 7fecea59c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fecec06ade0 0x7fecec0fc4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53656/0 (socket says 172.21.15.64:53656) 2024-02-20T12:50:15.369 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:15.363+0000 7fecea59c700 1 -- 172.21.15.64:0/221592007 learned_addr learned my addr 172.21.15.64:0/221592007 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-20T12:50:15.369 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:15.363+0000 7fecea59c700 1 -- 172.21.15.64:0/221592007 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecdc00b200 con 0x7fecec06ade0 2024-02-20T12:50:15.369 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:15.363+0000 7fecea59c700 1 --2- 172.21.15.64:0/221592007 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fecec06ade0 0x7fecec0fc4b0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 rx=0x7fecdc000c00 tx=0x7fecdc006ee0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:50:15.369 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:15.363+0000 7fece2ffd700 1 -- 172.21.15.64:0/221592007 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fecdc01b410 con 0x7fecec06ade0 2024-02-20T12:50:15.369 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:15.363+0000 7fece2ffd700 1 -- 172.21.15.64:0/221592007 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(25 keys) v1 ==== 997+0+0 (secure 0 0 0) 0x7fecdc01b9d0 con 0x7fecec06ade0 2024-02-20T12:50:15.369 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:15.363+0000 7feceb59e700 1 -- 172.21.15.64:0/221592007 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fecec0faad0 con 0x7fecec06ade0 2024-02-20T12:50:15.369 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:15.363+0000 7fece2ffd700 1 -- 172.21.15.64:0/221592007 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fecdc01a620 con 0x7fecec06ade0 2024-02-20T12:50:15.369 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:15.363+0000 7feceb59e700 1 -- 172.21.15.64:0/221592007 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fecec0faf60 con 0x7fecec06ade0 2024-02-20T12:50:15.369 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:15.364+0000 7fece2ffd700 1 -- 172.21.15.64:0/221592007 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 4) v1 ==== 37084+0+0 (secure 0 0 0) 0x7fecdc02b070 con 0x7fecec06ade0 2024-02-20T12:50:15.369 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:15.364+0000 7fece2ffd700 1 --2- 172.21.15.64:0/221592007 >> [v2:172.21.15.64:6800/3845912330,v1:172.21.15.64:6801/3845912330] conn(0x7feccc030fa0 0x7feccc033460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:50:15.369 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:15.364+0000 7fece2ffd700 1 -- 172.21.15.64:0/221592007 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7fecdc042a90 con 0x7fecec06ade0 2024-02-20T12:50:15.369 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:15.364+0000 7fece9d9b700 1 --2- 172.21.15.64:0/221592007 >> [v2:172.21.15.64:6800/3845912330,v1:172.21.15.64:6801/3845912330] conn(0x7feccc030fa0 0x7feccc033460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:50:15.369 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:15.365+0000 7feceb59e700 1 -- 172.21.15.64:0/221592007 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fecec0591e0 con 0x7fecec06ade0 2024-02-20T12:50:15.369 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:15.365+0000 7fece9d9b700 1 --2- 172.21.15.64:0/221592007 >> [v2:172.21.15.64:6800/3845912330,v1:172.21.15.64:6801/3845912330] conn(0x7feccc030fa0 0x7feccc033460 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 rx=0x7fecd40096a0 tx=0x7fecd4006b40).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:50:15.370 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:15.369+0000 7fece2ffd700 1 -- 172.21.15.64:0/221592007 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+153237 (secure 0 0 0) 0x7fecdc016020 con 0x7fecec06ade0 2024-02-20T12:50:15.530 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:15.529+0000 7feceb59e700 1 -- 172.21.15.64:0/221592007 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7fecec045bb0 con 0x7fecec06ade0 2024-02-20T12:50:15.534 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:15.530+0000 7fece2ffd700 1 -- 172.21.15.64:0/221592007 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) v1 ==== 70+0+398 (secure 0 0 0) 0x7fecdc01f070 con 0x7fecec06ade0 2024-02-20T12:50:15.534 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:15.534 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: [global] 2024-02-20T12:50:15.534 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: fsid = 680ee126-cfee-11ee-95bb-87774f69a715 2024-02-20T12:50:15.534 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: log_to_journald = false 2024-02-20T12:50:15.534 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: mon_osd_allow_pg_remap = true 2024-02-20T12:50:15.534 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: mon_osd_allow_primary_affinity = true 2024-02-20T12:50:15.535 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: mon_warn_on_no_sortbitwise = false 2024-02-20T12:50:15.535 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: osd_crush_chooseleaf_type = 0 2024-02-20T12:50:15.535 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:15.535 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: [mgr] 2024-02-20T12:50:15.535 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: mgr/telemetry/nag = false 2024-02-20T12:50:15.535 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:15.535 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: [osd] 2024-02-20T12:50:15.535 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: mon_osd_backfillfull_ratio = 0.85 2024-02-20T12:50:15.535 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: mon_osd_full_ratio = 0.9 2024-02-20T12:50:15.535 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: mon_osd_nearfull_ratio = 0.8 2024-02-20T12:50:15.535 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: osd_map_max_advance = 10 2024-02-20T12:50:15.535 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: osd_sloppy_crc = true 2024-02-20T12:50:15.535 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:15.532+0000 7feceb59e700 1 -- 172.21.15.64:0/221592007 >> [v2:172.21.15.64:6800/3845912330,v1:172.21.15.64:6801/3845912330] conn(0x7feccc030fa0 msgr2=0x7feccc033460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:50:15.535 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:15.532+0000 7feceb59e700 1 --2- 172.21.15.64:0/221592007 >> [v2:172.21.15.64:6800/3845912330,v1:172.21.15.64:6801/3845912330] conn(0x7feccc030fa0 0x7feccc033460 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 rx=0x7fecd40096a0 tx=0x7fecd4006b40).stop 2024-02-20T12:50:15.535 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:15.532+0000 7feceb59e700 1 -- 172.21.15.64:0/221592007 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fecec06ade0 msgr2=0x7fecec0fc4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:50:15.536 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:15.532+0000 7feceb59e700 1 --2- 172.21.15.64:0/221592007 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fecec06ade0 0x7fecec0fc4b0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 rx=0x7fecdc000c00 tx=0x7fecdc006ee0).stop 2024-02-20T12:50:15.536 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:15.532+0000 7feceb59e700 1 -- 172.21.15.64:0/221592007 shutdown_connections 2024-02-20T12:50:15.536 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:15.532+0000 7feceb59e700 1 --2- 172.21.15.64:0/221592007 >> [v2:172.21.15.64:6800/3845912330,v1:172.21.15.64:6801/3845912330] conn(0x7feccc030fa0 0x7feccc033460 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:50:15.536 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:15.532+0000 7feceb59e700 1 --2- 172.21.15.64:0/221592007 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fecec06ade0 0x7fecec0fc4b0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:50:15.536 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:15.532+0000 7feceb59e700 1 -- 172.21.15.64:0/221592007 >> 172.21.15.64:0/221592007 conn(0x7fecec0f64d0 msgr2=0x7fecec0f7f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:50:15.536 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:15.533+0000 7feceb59e700 1 -- 172.21.15.64:0/221592007 shutdown_connections 2024-02-20T12:50:15.536 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:15.533+0000 7feceb59e700 1 -- 172.21.15.64:0/221592007 wait complete. 2024-02-20T12:50:15.812 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:15 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:50:15.530963+0000 mon.smithi064 (mon.0) 25 : audit [INF] from='client.? 172.21.15.64:0/221592007' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-02-20T12:50:16.182 INFO:teuthology.orchestra.run.smithi064.stderr:Enabling cephadm module... 2024-02-20T12:50:16.797 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:16.791+0000 7f4bccd39700 1 Processor -- start 2024-02-20T12:50:16.797 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:16.791+0000 7f4bccd39700 1 -- start start 2024-02-20T12:50:16.797 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:16.792+0000 7f4bccd39700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4bc806ade0 0x7f4bc80febe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-02-20T12:50:16.797 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:16.792+0000 7f4bccd39700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bc80ff0d0 con 0x7f4bc806ade0 2024-02-20T12:50:16.797 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:16.792+0000 7f4bc659c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4bc806ade0 0x7f4bc80febe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:50:16.797 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:16.792+0000 7f4bc659c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4bc806ade0 0x7f4bc80febe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53670/0 (socket says 172.21.15.64:53670) 2024-02-20T12:50:16.798 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:16.792+0000 7f4bc659c700 1 -- 172.21.15.64:0/399895296 learned_addr learned my addr 172.21.15.64:0/399895296 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-20T12:50:16.798 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:16.792+0000 7f4bc659c700 1 -- 172.21.15.64:0/399895296 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bc80ff210 con 0x7f4bc806ade0 2024-02-20T12:50:16.798 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:16.792+0000 7f4bc659c700 1 --2- 172.21.15.64:0/399895296 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4bc806ade0 0x7f4bc80febe0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 rx=0x7f4bb800b5f0 tx=0x7f4bb8006ee0).ready entity=mon.0 client_cookie=13b4de8bc02a4e56 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:50:16.798 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:16.792+0000 7f4bc559a700 1 -- 172.21.15.64:0/399895296 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4bb800d600 con 0x7f4bc806ade0 2024-02-20T12:50:16.798 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:16.792+0000 7f4bc559a700 1 -- 172.21.15.64:0/399895296 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(25 keys) v1 ==== 997+0+0 (secure 0 0 0) 0x7f4bb800dbc0 con 0x7f4bc806ade0 2024-02-20T12:50:16.798 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:16.792+0000 7f4bc559a700 1 -- 172.21.15.64:0/399895296 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4bb8011b90 con 0x7f4bc806ade0 2024-02-20T12:50:16.798 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:16.793+0000 7f4bccd39700 1 -- 172.21.15.64:0/399895296 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4bc806ade0 msgr2=0x7f4bc80febe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:50:16.798 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:16.793+0000 7f4bccd39700 1 --2- 172.21.15.64:0/399895296 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4bc806ade0 0x7f4bc80febe0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 rx=0x7f4bb800b5f0 tx=0x7f4bb8006ee0).stop 2024-02-20T12:50:16.798 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:16.793+0000 7f4bccd39700 1 -- 172.21.15.64:0/399895296 shutdown_connections 2024-02-20T12:50:16.798 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:16.793+0000 7f4bccd39700 1 --2- 172.21.15.64:0/399895296 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4bc806ade0 0x7f4bc80febe0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:50:16.798 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:16.793+0000 7f4bccd39700 1 -- 172.21.15.64:0/399895296 >> 172.21.15.64:0/399895296 conn(0x7f4bc80f6680 msgr2=0x7f4bc80f8aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:50:16.798 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:16.793+0000 7f4bccd39700 1 -- 172.21.15.64:0/399895296 shutdown_connections 2024-02-20T12:50:16.798 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:16.793+0000 7f4bccd39700 1 -- 172.21.15.64:0/399895296 wait complete. 2024-02-20T12:50:16.799 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:16.794+0000 7f4bccd39700 1 Processor -- start 2024-02-20T12:50:16.799 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:16.794+0000 7f4bccd39700 1 -- start start 2024-02-20T12:50:16.799 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:16.794+0000 7f4bccd39700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4bc806ade0 0x7f4bc810ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:50:16.799 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:16.794+0000 7f4bccd39700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bc810b230 con 0x7f4bc806ade0 2024-02-20T12:50:16.799 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:16.794+0000 7f4bc659c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4bc806ade0 0x7f4bc810ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:50:16.799 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:16.794+0000 7f4bc659c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4bc806ade0 0x7f4bc810ad40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53684/0 (socket says 172.21.15.64:53684) 2024-02-20T12:50:16.799 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:16.794+0000 7f4bc659c700 1 -- 172.21.15.64:0/858743646 learned_addr learned my addr 172.21.15.64:0/858743646 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-20T12:50:16.799 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:16.794+0000 7f4bc659c700 1 -- 172.21.15.64:0/858743646 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bb800b200 con 0x7f4bc806ade0 2024-02-20T12:50:16.799 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:16.794+0000 7f4bc659c700 1 --2- 172.21.15.64:0/858743646 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4bc806ade0 0x7f4bc810ad40 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 rx=0x7f4bb8006800 tx=0x7f4bb8006ee0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:50:16.799 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:16.795+0000 7f4bbeffd700 1 -- 172.21.15.64:0/858743646 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4bb800de50 con 0x7f4bc806ade0 2024-02-20T12:50:16.799 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:16.795+0000 7f4bbeffd700 1 -- 172.21.15.64:0/858743646 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(25 keys) v1 ==== 997+0+0 (secure 0 0 0) 0x7f4bb801b870 con 0x7f4bc806ade0 2024-02-20T12:50:16.799 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:16.795+0000 7f4bccd39700 1 -- 172.21.15.64:0/858743646 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4bc810b430 con 0x7f4bc806ade0 2024-02-20T12:50:16.799 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:16.795+0000 7f4bbeffd700 1 -- 172.21.15.64:0/858743646 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4bb801a7a0 con 0x7f4bc806ade0 2024-02-20T12:50:16.799 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:16.795+0000 7f4bccd39700 1 -- 172.21.15.64:0/858743646 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4bc810c040 con 0x7f4bc806ade0 2024-02-20T12:50:16.799 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:16.796+0000 7f4bccd39700 1 -- 172.21.15.64:0/858743646 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4bc80591e0 con 0x7f4bc806ade0 2024-02-20T12:50:16.800 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:16.797+0000 7f4bbeffd700 1 -- 172.21.15.64:0/858743646 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 4) v1 ==== 37084+0+0 (secure 0 0 0) 0x7f4bb800f070 con 0x7f4bc806ade0 2024-02-20T12:50:16.800 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:16.797+0000 7f4bbeffd700 1 --2- 172.21.15.64:0/858743646 >> [v2:172.21.15.64:6800/3845912330,v1:172.21.15.64:6801/3845912330] conn(0x7f4ba8030fa0 0x7f4ba8033460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:50:16.800 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:16.797+0000 7f4bbeffd700 1 -- 172.21.15.64:0/858743646 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f4bb8043420 con 0x7f4bc806ade0 2024-02-20T12:50:16.800 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:16.797+0000 7f4bc5d9b700 1 --2- 172.21.15.64:0/858743646 >> [v2:172.21.15.64:6800/3845912330,v1:172.21.15.64:6801/3845912330] conn(0x7f4ba8030fa0 0x7f4ba8033460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:50:16.801 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:16.797+0000 7f4bc5d9b700 1 --2- 172.21.15.64:0/858743646 >> [v2:172.21.15.64:6800/3845912330,v1:172.21.15.64:6801/3845912330] conn(0x7f4ba8030fa0 0x7f4ba8033460 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0x7f4bb00096a0 tx=0x7f4bb0006b40).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:50:16.802 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:16.801+0000 7f4bbeffd700 1 -- 172.21.15.64:0/858743646 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+153237 (secure 0 0 0) 0x7f4bb8022360 con 0x7f4bc806ade0 2024-02-20T12:50:17.004 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:17.003+0000 7f4bccd39700 1 -- 172.21.15.64:0/858743646 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) v1 -- 0x7f4bc8100090 con 0x7f4bc806ade0 2024-02-20T12:50:17.056 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:17.055+0000 7f4bbeffd700 1 -- 172.21.15.64:0/858743646 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mgrmap(e 5) v1 ==== 37095+0+0 (secure 0 0 0) 0x7f4bb801b9e0 con 0x7f4bc806ade0 2024-02-20T12:50:17.057 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:17.056+0000 7f4bbeffd700 1 -- 172.21.15.64:0/858743646 <== mon.0 v2:172.21.15.64:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) v1 ==== 86+0+0 (secure 0 0 0) 0x7f4bb802de30 con 0x7f4bc806ade0 2024-02-20T12:50:17.060 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:17.058+0000 7f4bccd39700 1 -- 172.21.15.64:0/858743646 >> [v2:172.21.15.64:6800/3845912330,v1:172.21.15.64:6801/3845912330] conn(0x7f4ba8030fa0 msgr2=0x7f4ba8033460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:50:17.060 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:17.058+0000 7f4bccd39700 1 --2- 172.21.15.64:0/858743646 >> [v2:172.21.15.64:6800/3845912330,v1:172.21.15.64:6801/3845912330] conn(0x7f4ba8030fa0 0x7f4ba8033460 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0x7f4bb00096a0 tx=0x7f4bb0006b40).stop 2024-02-20T12:50:17.060 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:17.058+0000 7f4bccd39700 1 -- 172.21.15.64:0/858743646 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4bc806ade0 msgr2=0x7f4bc810ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:50:17.060 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:17.058+0000 7f4bccd39700 1 --2- 172.21.15.64:0/858743646 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4bc806ade0 0x7f4bc810ad40 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 rx=0x7f4bb8006800 tx=0x7f4bb8006ee0).stop 2024-02-20T12:50:17.061 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:17.058+0000 7f4bccd39700 1 -- 172.21.15.64:0/858743646 shutdown_connections 2024-02-20T12:50:17.061 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:17.058+0000 7f4bccd39700 1 --2- 172.21.15.64:0/858743646 >> [v2:172.21.15.64:6800/3845912330,v1:172.21.15.64:6801/3845912330] conn(0x7f4ba8030fa0 0x7f4ba8033460 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:50:17.061 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:17.058+0000 7f4bccd39700 1 --2- 172.21.15.64:0/858743646 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4bc806ade0 0x7f4bc810ad40 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:50:17.061 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:17.058+0000 7f4bccd39700 1 -- 172.21.15.64:0/858743646 >> 172.21.15.64:0/858743646 conn(0x7f4bc80f6680 msgr2=0x7f4bc80f8150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:50:17.061 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:17.059+0000 7f4bccd39700 1 -- 172.21.15.64:0/858743646 shutdown_connections 2024-02-20T12:50:17.061 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:17.059+0000 7f4bccd39700 1 -- 172.21.15.64:0/858743646 wait complete. 2024-02-20T12:50:17.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:17 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:50:17.004724+0000 mon.smithi064 (mon.0) 26 : audit [INF] from='client.? 172.21.15.64:0/858743646' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-02-20T12:50:18.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:18 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:50:17.057852+0000 mon.smithi064 (mon.0) 27 : audit [INF] from='client.? 172.21.15.64:0/858743646' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-02-20T12:50:18.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:18 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:50:17.057907+0000 mon.smithi064 (mon.0) 28 : cluster [DBG] mgrmap e5: smithi064.vriyor(active, since 5s) 2024-02-20T12:50:18.351 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:18.313+0000 7f746759e700 1 Processor -- start 2024-02-20T12:50:18.351 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:18.313+0000 7f746759e700 1 -- start start 2024-02-20T12:50:18.351 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:18.314+0000 7f746759e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f74680fad70 0x7f74680fb190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-02-20T12:50:18.351 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:18.314+0000 7f746759e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74680fb680 con 0x7f74680fad70 2024-02-20T12:50:18.352 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:18.314+0000 7f746659c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f74680fad70 0x7f74680fb190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:50:18.352 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:18.314+0000 7f746659c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f74680fad70 0x7f74680fb190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53712/0 (socket says 172.21.15.64:53712) 2024-02-20T12:50:18.352 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:18.314+0000 7f746659c700 1 -- 172.21.15.64:0/4104331798 learned_addr learned my addr 172.21.15.64:0/4104331798 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-20T12:50:18.352 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:18.314+0000 7f746659c700 1 -- 172.21.15.64:0/4104331798 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74680fb7c0 con 0x7f74680fad70 2024-02-20T12:50:18.352 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:18.314+0000 7f746659c700 1 --2- 172.21.15.64:0/4104331798 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f74680fad70 0x7f74680fb190 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7f745800b5f0 tx=0x7f7458006ee0).ready entity=mon.0 client_cookie=1bde8ebcf16057df server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:50:18.352 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:18.315+0000 7f746559a700 1 -- 172.21.15.64:0/4104331798 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f745800d600 con 0x7f74680fad70 2024-02-20T12:50:18.352 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:18.315+0000 7f746559a700 1 -- 172.21.15.64:0/4104331798 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(25 keys) v1 ==== 997+0+0 (secure 0 0 0) 0x7f745800dbc0 con 0x7f74680fad70 2024-02-20T12:50:18.352 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:18.315+0000 7f746559a700 1 -- 172.21.15.64:0/4104331798 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7458011b90 con 0x7f74680fad70 2024-02-20T12:50:18.352 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:18.315+0000 7f746759e700 1 -- 172.21.15.64:0/4104331798 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f74680fad70 msgr2=0x7f74680fb190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:50:18.352 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:18.315+0000 7f746759e700 1 --2- 172.21.15.64:0/4104331798 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f74680fad70 0x7f74680fb190 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7f745800b5f0 tx=0x7f7458006ee0).stop 2024-02-20T12:50:18.352 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:18.315+0000 7f746759e700 1 -- 172.21.15.64:0/4104331798 shutdown_connections 2024-02-20T12:50:18.352 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:18.315+0000 7f746759e700 1 --2- 172.21.15.64:0/4104331798 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f74680fad70 0x7f74680fb190 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:50:18.353 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:18.315+0000 7f746759e700 1 -- 172.21.15.64:0/4104331798 >> 172.21.15.64:0/4104331798 conn(0x7f74680f64d0 msgr2=0x7f74680f88f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:50:18.353 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:18.315+0000 7f746759e700 1 -- 172.21.15.64:0/4104331798 shutdown_connections 2024-02-20T12:50:18.353 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:18.315+0000 7f746759e700 1 -- 172.21.15.64:0/4104331798 wait complete. 2024-02-20T12:50:18.353 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:18.316+0000 7f746759e700 1 Processor -- start 2024-02-20T12:50:18.353 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:18.316+0000 7f746759e700 1 -- start start 2024-02-20T12:50:18.353 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:18.316+0000 7f746759e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f74680fad70 0x7f7468109a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:50:18.353 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:18.316+0000 7f746759e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7468109ef0 con 0x7f74680fad70 2024-02-20T12:50:18.353 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:18.316+0000 7f746659c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f74680fad70 0x7f7468109a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:50:18.353 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:18.316+0000 7f746659c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f74680fad70 0x7f7468109a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53724/0 (socket says 172.21.15.64:53724) 2024-02-20T12:50:18.353 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:18.316+0000 7f746659c700 1 -- 172.21.15.64:0/1145521205 learned_addr learned my addr 172.21.15.64:0/1145521205 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-20T12:50:18.353 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:18.316+0000 7f746659c700 1 -- 172.21.15.64:0/1145521205 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f745800b200 con 0x7f74680fad70 2024-02-20T12:50:18.353 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:18.317+0000 7f746659c700 1 --2- 172.21.15.64:0/1145521205 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f74680fad70 0x7f7468109a00 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7f7458000c00 tx=0x7f7458006ee0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:50:18.353 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:18.317+0000 7f7456ffd700 1 -- 172.21.15.64:0/1145521205 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f745801b410 con 0x7f74680fad70 2024-02-20T12:50:18.353 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:18.317+0000 7f746759e700 1 -- 172.21.15.64:0/1145521205 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f746810a0f0 con 0x7f74680fad70 2024-02-20T12:50:18.353 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:18.317+0000 7f746759e700 1 -- 172.21.15.64:0/1145521205 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f746810a580 con 0x7f74680fad70 2024-02-20T12:50:18.354 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:18.317+0000 7f7456ffd700 1 -- 172.21.15.64:0/1145521205 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(25 keys) v1 ==== 997+0+0 (secure 0 0 0) 0x7f745801b9d0 con 0x7f74680fad70 2024-02-20T12:50:18.354 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:18.317+0000 7f7456ffd700 1 -- 172.21.15.64:0/1145521205 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f745801a600 con 0x7f74680fad70 2024-02-20T12:50:18.354 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:18.318+0000 7f7456ffd700 1 -- 172.21.15.64:0/1145521205 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 37095+0+0 (secure 0 0 0) 0x7f745802b070 con 0x7f74680fad70 2024-02-20T12:50:18.354 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:18.318+0000 7f7456ffd700 1 --2- 172.21.15.64:0/1145521205 >> [v2:172.21.15.64:6800/3845912330,v1:172.21.15.64:6801/3845912330] conn(0x7f744c031040 0x7f744c033500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:50:18.354 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:18.319+0000 7f7456ffd700 1 -- 172.21.15.64:0/1145521205 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f7458043910 con 0x7f74680fad70 2024-02-20T12:50:18.354 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:18.319+0000 7f7465d9b700 1 -- 172.21.15.64:0/1145521205 >> [v2:172.21.15.64:6800/3845912330,v1:172.21.15.64:6801/3845912330] conn(0x7f744c031040 msgr2=0x7f744c033500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.64:6800/3845912330 2024-02-20T12:50:18.354 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:18.319+0000 7f7465d9b700 1 --2- 172.21.15.64:0/1145521205 >> [v2:172.21.15.64:6800/3845912330,v1:172.21.15.64:6801/3845912330] conn(0x7f744c031040 0x7f744c033500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2024-02-20T12:50:18.354 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:18.320+0000 7f746759e700 1 -- 172.21.15.64:0/1145521205 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7468045bb0 con 0x7f74680fad70 2024-02-20T12:50:18.354 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:18.329+0000 7f7456ffd700 1 -- 172.21.15.64:0/1145521205 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+153237 (secure 0 0 0) 0x7f7458016080 con 0x7f74680fad70 2024-02-20T12:50:18.521 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:18.520+0000 7f7465d9b700 1 -- 172.21.15.64:0/1145521205 >> [v2:172.21.15.64:6800/3845912330,v1:172.21.15.64:6801/3845912330] conn(0x7f744c031040 msgr2=0x7f744c033500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.64:6800/3845912330 2024-02-20T12:50:18.521 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:18.520+0000 7f7465d9b700 1 --2- 172.21.15.64:0/1145521205 >> [v2:172.21.15.64:6800/3845912330,v1:172.21.15.64:6801/3845912330] conn(0x7f744c031040 0x7f744c033500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2024-02-20T12:50:18.572 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:18.570+0000 7f746759e700 1 -- 172.21.15.64:0/1145521205 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7f7468103a90 con 0x7f74680fad70 2024-02-20T12:50:18.572 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:18.571+0000 7f7456ffd700 1 -- 172.21.15.64:0/1145521205 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) v1 ==== 56+0+103 (secure 0 0 0) 0x7f745801f020 con 0x7f74680fad70 2024-02-20T12:50:18.572 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: { 2024-02-20T12:50:18.573 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: "epoch": 5, 2024-02-20T12:50:18.573 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: "available": true, 2024-02-20T12:50:18.573 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: "active_name": "smithi064.vriyor", 2024-02-20T12:50:18.573 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: "num_standby": 0 2024-02-20T12:50:18.573 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: } 2024-02-20T12:50:18.574 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:18.572+0000 7f746759e700 1 -- 172.21.15.64:0/1145521205 >> [v2:172.21.15.64:6800/3845912330,v1:172.21.15.64:6801/3845912330] conn(0x7f744c031040 msgr2=0x7f744c033500 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-20T12:50:18.574 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:18.572+0000 7f746759e700 1 --2- 172.21.15.64:0/1145521205 >> [v2:172.21.15.64:6800/3845912330,v1:172.21.15.64:6801/3845912330] conn(0x7f744c031040 0x7f744c033500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-02-20T12:50:18.575 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:18.572+0000 7f746759e700 1 -- 172.21.15.64:0/1145521205 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f74680fad70 msgr2=0x7f7468109a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:50:18.575 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:18.572+0000 7f746759e700 1 --2- 172.21.15.64:0/1145521205 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f74680fad70 0x7f7468109a00 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7f7458000c00 tx=0x7f7458006ee0).stop 2024-02-20T12:50:18.575 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:18.572+0000 7f746759e700 1 -- 172.21.15.64:0/1145521205 shutdown_connections 2024-02-20T12:50:18.575 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:18.573+0000 7f746759e700 1 --2- 172.21.15.64:0/1145521205 >> [v2:172.21.15.64:6800/3845912330,v1:172.21.15.64:6801/3845912330] conn(0x7f744c031040 0x7f744c033500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-02-20T12:50:18.575 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:18.573+0000 7f746759e700 1 --2- 172.21.15.64:0/1145521205 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f74680fad70 0x7f7468109a00 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:50:18.575 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:18.573+0000 7f746759e700 1 -- 172.21.15.64:0/1145521205 >> 172.21.15.64:0/1145521205 conn(0x7f74680f64d0 msgr2=0x7f74680f7f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:50:18.575 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:18.573+0000 7f746759e700 1 -- 172.21.15.64:0/1145521205 shutdown_connections 2024-02-20T12:50:18.575 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:18.573+0000 7f746759e700 1 -- 172.21.15.64:0/1145521205 wait complete. 2024-02-20T12:50:19.287 INFO:teuthology.orchestra.run.smithi064.stderr:Waiting for the mgr to restart... 2024-02-20T12:50:19.287 INFO:teuthology.orchestra.run.smithi064.stderr:Waiting for mgr epoch 5... 2024-02-20T12:50:19.289 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:19 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:50:18.571918+0000 mon.smithi064 (mon.0) 29 : audit [DBG] from='client.? 172.21.15.64:0/1145521205' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-02-20T12:50:19.909 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:19.862+0000 7f589d86c700 1 Processor -- start 2024-02-20T12:50:19.909 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:19.862+0000 7f589d86c700 1 -- start start 2024-02-20T12:50:19.909 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:19.862+0000 7f589d86c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f589806ad60 0x7f5898100eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-02-20T12:50:19.909 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:19.862+0000 7f589d86c700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58980fc8a0 con 0x7f589806ad60 2024-02-20T12:50:19.909 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:19.862+0000 7f589c86a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f589806ad60 0x7f5898100eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:50:19.909 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:19.862+0000 7f589c86a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f589806ad60 0x7f5898100eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53728/0 (socket says 172.21.15.64:53728) 2024-02-20T12:50:19.909 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:19.862+0000 7f589c86a700 1 -- 172.21.15.64:0/3349761448 learned_addr learned my addr 172.21.15.64:0/3349761448 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-20T12:50:19.909 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:19.863+0000 7f589c86a700 1 -- 172.21.15.64:0/3349761448 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58981013a0 con 0x7f589806ad60 2024-02-20T12:50:19.909 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:19.863+0000 7f589c86a700 1 --2- 172.21.15.64:0/3349761448 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f589806ad60 0x7f5898100eb0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7f588c00b5f0 tx=0x7f588c006ee0).ready entity=mon.0 client_cookie=3de6b134715dd978 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:50:19.909 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:19.863+0000 7f58977fe700 1 -- 172.21.15.64:0/3349761448 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f588c00d600 con 0x7f589806ad60 2024-02-20T12:50:19.909 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:19.863+0000 7f58977fe700 1 -- 172.21.15.64:0/3349761448 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(25 keys) v1 ==== 997+0+0 (secure 0 0 0) 0x7f588c00dbc0 con 0x7f589806ad60 2024-02-20T12:50:19.910 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:19.863+0000 7f58977fe700 1 -- 172.21.15.64:0/3349761448 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f588c011b90 con 0x7f589806ad60 2024-02-20T12:50:19.910 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:19.863+0000 7f589d86c700 1 -- 172.21.15.64:0/3349761448 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f589806ad60 msgr2=0x7f5898100eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:50:19.910 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:19.863+0000 7f589d86c700 1 --2- 172.21.15.64:0/3349761448 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f589806ad60 0x7f5898100eb0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7f588c00b5f0 tx=0x7f588c006ee0).stop 2024-02-20T12:50:19.910 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:19.864+0000 7f589d86c700 1 -- 172.21.15.64:0/3349761448 shutdown_connections 2024-02-20T12:50:19.910 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:19.864+0000 7f589d86c700 1 --2- 172.21.15.64:0/3349761448 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f589806ad60 0x7f5898100eb0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:50:19.910 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:19.864+0000 7f589d86c700 1 -- 172.21.15.64:0/3349761448 >> 172.21.15.64:0/3349761448 conn(0x7f58980f64d0 msgr2=0x7f58980f88f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:50:19.910 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:19.864+0000 7f589d86c700 1 -- 172.21.15.64:0/3349761448 shutdown_connections 2024-02-20T12:50:19.910 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:19.864+0000 7f589d86c700 1 -- 172.21.15.64:0/3349761448 wait complete. 2024-02-20T12:50:19.910 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:19.864+0000 7f589d86c700 1 Processor -- start 2024-02-20T12:50:19.910 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:19.864+0000 7f589d86c700 1 -- start start 2024-02-20T12:50:19.910 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:19.864+0000 7f589d86c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f589806ad60 0x7f589810cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:50:19.910 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:19.864+0000 7f589d86c700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f589810d2a0 con 0x7f589806ad60 2024-02-20T12:50:19.910 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:19.865+0000 7f589c86a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f589806ad60 0x7f589810cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:50:19.910 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:19.865+0000 7f589c86a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f589806ad60 0x7f589810cdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53736/0 (socket says 172.21.15.64:53736) 2024-02-20T12:50:19.910 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:19.865+0000 7f589c86a700 1 -- 172.21.15.64:0/4135463118 learned_addr learned my addr 172.21.15.64:0/4135463118 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-20T12:50:19.911 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:19.865+0000 7f589c86a700 1 -- 172.21.15.64:0/4135463118 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f588c00b200 con 0x7f589806ad60 2024-02-20T12:50:19.911 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:19.865+0000 7f589c86a700 1 --2- 172.21.15.64:0/4135463118 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f589806ad60 0x7f589810cdb0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7f588c006800 tx=0x7f588c006ee0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:50:19.911 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:19.865+0000 7f58957fa700 1 -- 172.21.15.64:0/4135463118 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f588c00d9c0 con 0x7f589806ad60 2024-02-20T12:50:19.911 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:19.865+0000 7f589d86c700 1 -- 172.21.15.64:0/4135463118 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f589810d4a0 con 0x7f589806ad60 2024-02-20T12:50:19.911 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:19.865+0000 7f58957fa700 1 -- 172.21.15.64:0/4135463118 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(25 keys) v1 ==== 997+0+0 (secure 0 0 0) 0x7f588c01b870 con 0x7f589806ad60 2024-02-20T12:50:19.911 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:19.865+0000 7f58957fa700 1 -- 172.21.15.64:0/4135463118 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f588c01a740 con 0x7f589806ad60 2024-02-20T12:50:19.911 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:19.865+0000 7f589d86c700 1 -- 172.21.15.64:0/4135463118 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f589810e0e0 con 0x7f589806ad60 2024-02-20T12:50:19.911 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:19.866+0000 7f58957fa700 1 -- 172.21.15.64:0/4135463118 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 5) v1 ==== 37095+0+0 (secure 0 0 0) 0x7f588c02b070 con 0x7f589806ad60 2024-02-20T12:50:19.911 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:19.866+0000 7f58957fa700 1 --2- 172.21.15.64:0/4135463118 >> [v2:172.21.15.64:6800/3845912330,v1:172.21.15.64:6801/3845912330] conn(0x7f587c030fa0 0x7f587c033460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:50:19.911 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:19.866+0000 7f58957fa700 1 -- 172.21.15.64:0/4135463118 --> [v2:172.21.15.64:6800/3845912330,v1:172.21.15.64:6801/3845912330] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f588c011610 con 0x7f587c030fa0 2024-02-20T12:50:19.911 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:19.866+0000 7f58957fa700 1 -- 172.21.15.64:0/4135463118 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f588c042c20 con 0x7f589806ad60 2024-02-20T12:50:19.911 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:19.866+0000 7f5897fff700 1 -- 172.21.15.64:0/4135463118 >> [v2:172.21.15.64:6800/3845912330,v1:172.21.15.64:6801/3845912330] conn(0x7f587c030fa0 msgr2=0x7f587c033460 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.64:6800/3845912330 2024-02-20T12:50:19.911 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:19.866+0000 7f5897fff700 1 --2- 172.21.15.64:0/4135463118 >> [v2:172.21.15.64:6800/3845912330,v1:172.21.15.64:6801/3845912330] conn(0x7f587c030fa0 0x7f587c033460 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2024-02-20T12:50:20.068 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:20.067+0000 7f5897fff700 1 -- 172.21.15.64:0/4135463118 >> [v2:172.21.15.64:6800/3845912330,v1:172.21.15.64:6801/3845912330] conn(0x7f587c030fa0 msgr2=0x7f587c033460 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.64:6800/3845912330 2024-02-20T12:50:20.068 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:20.067+0000 7f5897fff700 1 --2- 172.21.15.64:0/4135463118 >> [v2:172.21.15.64:6800/3845912330,v1:172.21.15.64:6801/3845912330] conn(0x7f587c030fa0 0x7f587c033460 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2024-02-20T12:50:20.469 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:20.468+0000 7f5897fff700 1 -- 172.21.15.64:0/4135463118 >> [v2:172.21.15.64:6800/3845912330,v1:172.21.15.64:6801/3845912330] conn(0x7f587c030fa0 msgr2=0x7f587c033460 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.64:6800/3845912330 2024-02-20T12:50:20.469 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:20.468+0000 7f5897fff700 1 --2- 172.21.15.64:0/4135463118 >> [v2:172.21.15.64:6800/3845912330,v1:172.21.15.64:6801/3845912330] conn(0x7f587c030fa0 0x7f587c033460 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.800000 2024-02-20T12:50:21.270 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:21.269+0000 7f5897fff700 1 -- 172.21.15.64:0/4135463118 >> [v2:172.21.15.64:6800/3845912330,v1:172.21.15.64:6801/3845912330] conn(0x7f587c030fa0 msgr2=0x7f587c033460 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.64:6800/3845912330 2024-02-20T12:50:21.270 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:21.269+0000 7f5897fff700 1 --2- 172.21.15.64:0/4135463118 >> [v2:172.21.15.64:6800/3845912330,v1:172.21.15.64:6801/3845912330] conn(0x7f587c030fa0 0x7f587c033460 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 1.600000 2024-02-20T12:50:22.422 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:22.421+0000 7f58957fa700 1 -- 172.21.15.64:0/4135463118 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mgrmap(e 6) v1 ==== 36945+0+0 (secure 0 0 0) 0x7f588c00f070 con 0x7f589806ad60 2024-02-20T12:50:22.423 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:22.421+0000 7f58957fa700 1 -- 172.21.15.64:0/4135463118 >> [v2:172.21.15.64:6800/3845912330,v1:172.21.15.64:6801/3845912330] conn(0x7f587c030fa0 msgr2=0x7f587c033460 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-20T12:50:22.423 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:22.421+0000 7f58957fa700 1 --2- 172.21.15.64:0/4135463118 >> [v2:172.21.15.64:6800/3845912330,v1:172.21.15.64:6801/3845912330] conn(0x7f587c030fa0 0x7f587c033460 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-02-20T12:50:22.812 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:22 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:50:22.369081+0000 mon.smithi064 (mon.0) 30 : cluster [INF] Active manager daemon smithi064.vriyor restarted 2024-02-20T12:50:22.812 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:22 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:50:22.369585+0000 mon.smithi064 (mon.0) 31 : cluster [INF] Activating manager daemon smithi064.vriyor 2024-02-20T12:50:22.812 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:22 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:50:22.373550+0000 mon.smithi064 (mon.0) 32 : cluster [DBG] osdmap e2: 0 total, 0 up, 0 in 2024-02-20T12:50:23.430 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:23.428+0000 7f58957fa700 1 -- 172.21.15.64:0/4135463118 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mgrmap(e 7) v1 ==== 37015+0+0 (secure 0 0 0) 0x7f588c01aaa0 con 0x7f589806ad60 2024-02-20T12:50:23.431 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:23.429+0000 7f58957fa700 1 --2- 172.21.15.64:0/4135463118 >> [v2:172.21.15.64:6800/1362908731,v1:172.21.15.64:6801/1362908731] conn(0x7f587c034890 0x7f587c036c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:50:23.431 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:23.429+0000 7f58957fa700 1 -- 172.21.15.64:0/4135463118 --> [v2:172.21.15.64:6800/1362908731,v1:172.21.15.64:6801/1362908731] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f588c011610 con 0x7f587c034890 2024-02-20T12:50:23.431 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:23.429+0000 7f5897fff700 1 --2- 172.21.15.64:0/4135463118 >> [v2:172.21.15.64:6800/1362908731,v1:172.21.15.64:6801/1362908731] conn(0x7f587c034890 0x7f587c036c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:50:23.431 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:23.430+0000 7f5897fff700 1 --2- 172.21.15.64:0/4135463118 >> [v2:172.21.15.64:6800/1362908731,v1:172.21.15.64:6801/1362908731] conn(0x7f587c034890 0x7f587c036c80 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 rx=0x7f58840036d0 tx=0x7f5884006ca0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:50:23.431 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:23.431+0000 7f58957fa700 1 -- 172.21.15.64:0/4135463118 <== mgr.14116 v2:172.21.15.64:6800/1362908731 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+6438 (secure 0 0 0) 0x7f588c011610 con 0x7f587c034890 2024-02-20T12:50:23.440 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:23.437+0000 7f589d86c700 1 -- 172.21.15.64:0/4135463118 --> [v2:172.21.15.64:6800/1362908731,v1:172.21.15.64:6801/1362908731] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7f589810d930 con 0x7f587c034890 2024-02-20T12:50:23.440 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:23.437+0000 7f58957fa700 1 -- 172.21.15.64:0/4135463118 <== mgr.14116 v2:172.21.15.64:6800/1362908731 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7f589810d930 con 0x7f587c034890 2024-02-20T12:50:23.440 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: { 2024-02-20T12:50:23.440 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: "mgrmap_epoch": 7, 2024-02-20T12:50:23.440 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: "initialized": true 2024-02-20T12:50:23.440 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: } 2024-02-20T12:50:23.440 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:23.437+0000 7f589d86c700 1 -- 172.21.15.64:0/4135463118 >> [v2:172.21.15.64:6800/1362908731,v1:172.21.15.64:6801/1362908731] conn(0x7f587c034890 msgr2=0x7f587c036c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:50:23.440 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:23.437+0000 7f589d86c700 1 --2- 172.21.15.64:0/4135463118 >> [v2:172.21.15.64:6800/1362908731,v1:172.21.15.64:6801/1362908731] conn(0x7f587c034890 0x7f587c036c80 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 rx=0x7f58840036d0 tx=0x7f5884006ca0).stop 2024-02-20T12:50:23.443 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:23.437+0000 7f589d86c700 1 -- 172.21.15.64:0/4135463118 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f589806ad60 msgr2=0x7f589810cdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:50:23.443 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:23.437+0000 7f589d86c700 1 --2- 172.21.15.64:0/4135463118 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f589806ad60 0x7f589810cdb0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7f588c006800 tx=0x7f588c006ee0).stop 2024-02-20T12:50:23.443 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:23.438+0000 7f589d86c700 1 -- 172.21.15.64:0/4135463118 shutdown_connections 2024-02-20T12:50:23.443 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:23.438+0000 7f589d86c700 1 --2- 172.21.15.64:0/4135463118 >> [v2:172.21.15.64:6800/1362908731,v1:172.21.15.64:6801/1362908731] conn(0x7f587c034890 0x7f587c036c80 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:50:23.444 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:23.438+0000 7f589d86c700 1 --2- 172.21.15.64:0/4135463118 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f589806ad60 0x7f589810cdb0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:50:23.444 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:23.438+0000 7f589d86c700 1 --2- 172.21.15.64:0/4135463118 >> [v2:172.21.15.64:6800/3845912330,v1:172.21.15.64:6801/3845912330] conn(0x7f587c030fa0 0x7f587c033460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-02-20T12:50:23.444 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:23.438+0000 7f589d86c700 1 -- 172.21.15.64:0/4135463118 >> 172.21.15.64:0/4135463118 conn(0x7f58980f64d0 msgr2=0x7f58980f7fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:50:23.444 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:23.438+0000 7f589d86c700 1 -- 172.21.15.64:0/4135463118 shutdown_connections 2024-02-20T12:50:23.444 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:23.438+0000 7f589d86c700 1 -- 172.21.15.64:0/4135463118 wait complete. 2024-02-20T12:50:23.812 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:23 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:50:22.423052+0000 mon.smithi064 (mon.0) 33 : cluster [DBG] mgrmap e6: smithi064.vriyor(active, starting, since 0.053572s) 2024-02-20T12:50:23.812 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:23 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:50:22.425521+0000 mon.smithi064 (mon.0) 34 : audit [DBG] from='mgr.14116 172.21.15.64:0/330181513' entity='mgr.smithi064.vriyor' cmd=[{"prefix": "mon metadata", "id": "smithi064"}]: dispatch 2024-02-20T12:50:23.812 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:23 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:50:22.426188+0000 mon.smithi064 (mon.0) 35 : audit [DBG] from='mgr.14116 172.21.15.64:0/330181513' entity='mgr.smithi064.vriyor' cmd=[{"prefix": "mgr metadata", "who": "smithi064.vriyor", "id": "smithi064.vriyor"}]: dispatch 2024-02-20T12:50:23.812 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:23 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:50:22.427007+0000 mon.smithi064 (mon.0) 36 : audit [DBG] from='mgr.14116 172.21.15.64:0/330181513' entity='mgr.smithi064.vriyor' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-20T12:50:23.812 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:23 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:50:22.427155+0000 mon.smithi064 (mon.0) 37 : audit [DBG] from='mgr.14116 172.21.15.64:0/330181513' entity='mgr.smithi064.vriyor' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-20T12:50:23.812 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:23 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:50:22.427302+0000 mon.smithi064 (mon.0) 38 : audit [DBG] from='mgr.14116 172.21.15.64:0/330181513' entity='mgr.smithi064.vriyor' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-20T12:50:23.813 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:23 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:50:22.440264+0000 mon.smithi064 (mon.0) 39 : cluster [INF] Manager daemon smithi064.vriyor is now available 2024-02-20T12:50:23.813 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:23 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:50:22.453557+0000 mon.smithi064 (mon.0) 40 : audit [INF] from='mgr.14116 172.21.15.64:0/330181513' entity='mgr.smithi064.vriyor' 2024-02-20T12:50:23.813 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:23 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:50:22.456242+0000 mon.smithi064 (mon.0) 41 : audit [INF] from='mgr.14116 172.21.15.64:0/330181513' entity='mgr.smithi064.vriyor' 2024-02-20T12:50:23.813 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:23 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:50:22.458793+0000 mon.smithi064 (mon.0) 42 : audit [INF] from='mgr.14116 172.21.15.64:0/330181513' entity='mgr.smithi064.vriyor' 2024-02-20T12:50:23.813 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:23 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:50:22.551748+0000 mon.smithi064 (mon.0) 43 : audit [INF] from='mgr.14116 172.21.15.64:0/330181513' entity='mgr.smithi064.vriyor' 2024-02-20T12:50:23.813 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:23 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:50:22.552440+0000 mon.smithi064 (mon.0) 44 : audit [DBG] from='mgr.14116 172.21.15.64:0/330181513' entity='mgr.smithi064.vriyor' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-20T12:50:23.813 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:23 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:50:22.553736+0000 mon.smithi064 (mon.0) 45 : audit [DBG] from='mgr.14116 172.21.15.64:0/330181513' entity='mgr.smithi064.vriyor' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-20T12:50:23.813 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:23 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:50:22.555496+0000 mon.smithi064 (mon.0) 46 : audit [DBG] from='mgr.14116 172.21.15.64:0/330181513' entity='mgr.smithi064.vriyor' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-20T12:50:23.813 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:23 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:50:22.566104+0000 mon.smithi064 (mon.0) 47 : audit [INF] from='mgr.14116 172.21.15.64:0/330181513' entity='mgr.smithi064.vriyor' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi064.vriyor/mirror_snapshot_schedule"}]: dispatch 2024-02-20T12:50:23.813 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:23 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:50:22.567275+0000 mon.smithi064 (mon.0) 48 : audit [INF] from='mgr.14116 172.21.15.64:0/330181513' entity='mgr.smithi064.vriyor' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi064.vriyor/trash_purge_schedule"}]: dispatch 2024-02-20T12:50:24.186 INFO:teuthology.orchestra.run.smithi064.stderr:mgr epoch 5 is available 2024-02-20T12:50:24.186 INFO:teuthology.orchestra.run.smithi064.stderr:Setting orchestrator backend to cephadm... 2024-02-20T12:50:24.460 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:24 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:50:23.431619+0000 mgr.smithi064.vriyor (mgr.14116) 1 : audit [DBG] from='client.14120 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-02-20T12:50:24.460 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:24 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:50:23.432403+0000 mon.smithi064 (mon.0) 49 : cluster [DBG] mgrmap e7: smithi064.vriyor(active, since 1.06292s) 2024-02-20T12:50:24.460 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:24 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:50:23.438348+0000 mgr.smithi064.vriyor (mgr.14116) 2 : audit [DBG] from='client.14120 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-02-20T12:50:24.839 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:24.836+0000 7f930f155700 1 Processor -- start 2024-02-20T12:50:24.839 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:24.836+0000 7f930f155700 1 -- start start 2024-02-20T12:50:24.839 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:24.836+0000 7f930f155700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f930806ade0 0x7f93080febe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-02-20T12:50:24.839 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:24.836+0000 7f930f155700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93080ff0d0 con 0x7f930806ade0 2024-02-20T12:50:24.839 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:24.837+0000 7f930cef1700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f930806ade0 0x7f93080febe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:50:24.839 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:24.837+0000 7f930cef1700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f930806ade0 0x7f93080febe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:49466/0 (socket says 172.21.15.64:49466) 2024-02-20T12:50:24.840 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:24.837+0000 7f930cef1700 1 -- 172.21.15.64:0/164784897 learned_addr learned my addr 172.21.15.64:0/164784897 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-20T12:50:24.840 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:24.837+0000 7f930cef1700 1 -- 172.21.15.64:0/164784897 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93080ff210 con 0x7f930806ade0 2024-02-20T12:50:24.840 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:24.837+0000 7f930cef1700 1 --2- 172.21.15.64:0/164784897 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f930806ade0 0x7f93080febe0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 rx=0x7f92fc00b5f0 tx=0x7f92fc006ee0).ready entity=mon.0 client_cookie=916de9cf486d2533 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:50:24.840 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:24.837+0000 7f93077fe700 1 -- 172.21.15.64:0/164784897 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f92fc00d600 con 0x7f930806ade0 2024-02-20T12:50:24.840 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:24.837+0000 7f93077fe700 1 -- 172.21.15.64:0/164784897 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(25 keys) v1 ==== 997+0+0 (secure 0 0 0) 0x7f92fc00dbc0 con 0x7f930806ade0 2024-02-20T12:50:24.840 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:24.837+0000 7f93077fe700 1 -- 172.21.15.64:0/164784897 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f92fc011b90 con 0x7f930806ade0 2024-02-20T12:50:24.840 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:24.838+0000 7f930f155700 1 -- 172.21.15.64:0/164784897 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f930806ade0 msgr2=0x7f93080febe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:50:24.840 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:24.838+0000 7f930f155700 1 --2- 172.21.15.64:0/164784897 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f930806ade0 0x7f93080febe0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 rx=0x7f92fc00b5f0 tx=0x7f92fc006ee0).stop 2024-02-20T12:50:24.840 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:24.838+0000 7f930f155700 1 -- 172.21.15.64:0/164784897 shutdown_connections 2024-02-20T12:50:24.841 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:24.838+0000 7f930f155700 1 --2- 172.21.15.64:0/164784897 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f930806ade0 0x7f93080febe0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:50:24.841 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:24.838+0000 7f930f155700 1 -- 172.21.15.64:0/164784897 >> 172.21.15.64:0/164784897 conn(0x7f93080f6680 msgr2=0x7f93080f8aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:50:24.841 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:24.838+0000 7f930f155700 1 -- 172.21.15.64:0/164784897 shutdown_connections 2024-02-20T12:50:24.841 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:24.838+0000 7f930f155700 1 -- 172.21.15.64:0/164784897 wait complete. 2024-02-20T12:50:24.843 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:24.838+0000 7f930f155700 1 Processor -- start 2024-02-20T12:50:24.843 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:24.839+0000 7f930f155700 1 -- start start 2024-02-20T12:50:24.843 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:24.839+0000 7f930f155700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f930806ade0 0x7f93080fc660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:50:24.843 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:24.839+0000 7f930f155700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93080faae0 con 0x7f930806ade0 2024-02-20T12:50:24.843 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:24.839+0000 7f930cef1700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f930806ade0 0x7f93080fc660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:50:24.844 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:24.839+0000 7f930cef1700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f930806ade0 0x7f93080fc660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:49468/0 (socket says 172.21.15.64:49468) 2024-02-20T12:50:24.844 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:24.839+0000 7f930cef1700 1 -- 172.21.15.64:0/3461928020 learned_addr learned my addr 172.21.15.64:0/3461928020 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-20T12:50:24.844 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:24.839+0000 7f930cef1700 1 -- 172.21.15.64:0/3461928020 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92fc00b200 con 0x7f930806ade0 2024-02-20T12:50:24.844 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:24.839+0000 7f930cef1700 1 --2- 172.21.15.64:0/3461928020 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f930806ade0 0x7f93080fc660 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 rx=0x7f92fc006800 tx=0x7f92fc006ee0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:50:24.844 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:24.840+0000 7f93057fa700 1 -- 172.21.15.64:0/3461928020 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f92fc00de50 con 0x7f930806ade0 2024-02-20T12:50:24.844 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:24.840+0000 7f93057fa700 1 -- 172.21.15.64:0/3461928020 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(25 keys) v1 ==== 997+0+0 (secure 0 0 0) 0x7f92fc01b870 con 0x7f930806ade0 2024-02-20T12:50:24.844 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:24.840+0000 7f930f155700 1 -- 172.21.15.64:0/3461928020 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93080fac80 con 0x7f930806ade0 2024-02-20T12:50:24.844 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:24.840+0000 7f93057fa700 1 -- 172.21.15.64:0/3461928020 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f92fc01a7a0 con 0x7f930806ade0 2024-02-20T12:50:24.844 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:24.840+0000 7f930f155700 1 -- 172.21.15.64:0/3461928020 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93080fb110 con 0x7f930806ade0 2024-02-20T12:50:24.844 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:24.840+0000 7f93057fa700 1 -- 172.21.15.64:0/3461928020 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 7) v1 ==== 37015+0+0 (secure 0 0 0) 0x7f92fc00f070 con 0x7f930806ade0 2024-02-20T12:50:24.844 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:24.841+0000 7f93057fa700 1 --2- 172.21.15.64:0/3461928020 >> [v2:172.21.15.64:6800/1362908731,v1:172.21.15.64:6801/1362908731] conn(0x7f92ec030f00 0x7f92ec0333c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:50:24.845 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:24.841+0000 7f93057fa700 1 -- 172.21.15.64:0/3461928020 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 893+0+0 (secure 0 0 0) 0x7f92fc042b10 con 0x7f930806ade0 2024-02-20T12:50:24.845 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:24.841+0000 7f9307fff700 1 --2- 172.21.15.64:0/3461928020 >> [v2:172.21.15.64:6800/1362908731,v1:172.21.15.64:6801/1362908731] conn(0x7f92ec030f00 0x7f92ec0333c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:50:24.845 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:24.841+0000 7f9307fff700 1 --2- 172.21.15.64:0/3461928020 >> [v2:172.21.15.64:6800/1362908731,v1:172.21.15.64:6801/1362908731] conn(0x7f92ec030f00 0x7f92ec0333c0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 rx=0x7f92f40096a0 tx=0x7f92f4006b40).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:50:24.845 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:24.842+0000 7f930f155700 1 -- 172.21.15.64:0/3461928020 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93080591e0 con 0x7f930806ade0 2024-02-20T12:50:24.847 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:24.847+0000 7f93057fa700 1 -- 172.21.15.64:0/3461928020 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+153237 (secure 0 0 0) 0x7f92fc016030 con 0x7f930806ade0 2024-02-20T12:50:24.898 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:24.897+0000 7f93057fa700 1 -- 172.21.15.64:0/3461928020 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mgrmap(e 8) v1 ==== 37095+0+0 (secure 0 0 0) 0x7f92fc01b9e0 con 0x7f930806ade0 2024-02-20T12:50:25.019 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:25.018+0000 7f930f155700 1 -- 172.21.15.64:0/3461928020 --> [v2:172.21.15.64:6800/1362908731,v1:172.21.15.64:6801/1362908731] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) v1 -- 0x7f93080fff80 con 0x7f92ec030f00 2024-02-20T12:50:25.024 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:25.023+0000 7f93057fa700 1 -- 172.21.15.64:0/3461928020 <== mgr.14116 v2:172.21.15.64:6800/1362908731 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f93080fff80 con 0x7f92ec030f00 2024-02-20T12:50:25.027 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:25.025+0000 7f930f155700 1 -- 172.21.15.64:0/3461928020 >> [v2:172.21.15.64:6800/1362908731,v1:172.21.15.64:6801/1362908731] conn(0x7f92ec030f00 msgr2=0x7f92ec0333c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:50:25.027 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:25.025+0000 7f930f155700 1 --2- 172.21.15.64:0/3461928020 >> [v2:172.21.15.64:6800/1362908731,v1:172.21.15.64:6801/1362908731] conn(0x7f92ec030f00 0x7f92ec0333c0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 rx=0x7f92f40096a0 tx=0x7f92f4006b40).stop 2024-02-20T12:50:25.027 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:25.025+0000 7f930f155700 1 -- 172.21.15.64:0/3461928020 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f930806ade0 msgr2=0x7f93080fc660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:50:25.027 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:25.025+0000 7f930f155700 1 --2- 172.21.15.64:0/3461928020 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f930806ade0 0x7f93080fc660 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 rx=0x7f92fc006800 tx=0x7f92fc006ee0).stop 2024-02-20T12:50:25.027 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:25.025+0000 7f930f155700 1 -- 172.21.15.64:0/3461928020 shutdown_connections 2024-02-20T12:50:25.027 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:25.025+0000 7f930f155700 1 --2- 172.21.15.64:0/3461928020 >> [v2:172.21.15.64:6800/1362908731,v1:172.21.15.64:6801/1362908731] conn(0x7f92ec030f00 0x7f92ec0333c0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:50:25.027 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:25.025+0000 7f930f155700 1 --2- 172.21.15.64:0/3461928020 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f930806ade0 0x7f93080fc660 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:50:25.028 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:25.025+0000 7f930f155700 1 -- 172.21.15.64:0/3461928020 >> 172.21.15.64:0/3461928020 conn(0x7f93080f6680 msgr2=0x7f93080f8150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:50:25.028 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:25.025+0000 7f930f155700 1 -- 172.21.15.64:0/3461928020 shutdown_connections 2024-02-20T12:50:25.028 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:25.025+0000 7f930f155700 1 -- 172.21.15.64:0/3461928020 wait complete. 2024-02-20T12:50:25.973 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:25 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:50:24.899466+0000 mon.smithi064 (mon.0) 50 : cluster [DBG] mgrmap e8: smithi064.vriyor(active, since 2s) 2024-02-20T12:50:25.973 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:25 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:50:25.020462+0000 mgr.smithi064.vriyor (mgr.14116) 3 : audit [DBG] from='client.14126 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-02-20T12:50:25.973 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:25 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:50:25.024370+0000 mon.smithi064 (mon.0) 51 : audit [INF] from='mgr.14116 172.21.15.64:0/330181513' entity='mgr.smithi064.vriyor' 2024-02-20T12:50:25.973 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:25 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:50:25.028082+0000 mon.smithi064 (mon.0) 52 : audit [DBG] from='mgr.14116 172.21.15.64:0/330181513' entity='mgr.smithi064.vriyor' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-20T12:50:26.210 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:26.205+0000 7f3bb8fec700 1 Processor -- start 2024-02-20T12:50:26.210 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:26.205+0000 7f3bb8fec700 1 -- start start 2024-02-20T12:50:26.210 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:26.205+0000 7f3bb8fec700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3bb406ada0 0x7f3bb40fea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-02-20T12:50:26.210 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:26.205+0000 7f3bb8fec700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bb40fef60 con 0x7f3bb406ada0 2024-02-20T12:50:26.210 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:26.205+0000 7f3bb37fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3bb406ada0 0x7f3bb40fea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:50:26.210 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:26.205+0000 7f3bb37fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3bb406ada0 0x7f3bb40fea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:49472/0 (socket says 172.21.15.64:49472) 2024-02-20T12:50:26.210 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:26.205+0000 7f3bb37fe700 1 -- 172.21.15.64:0/1291273050 learned_addr learned my addr 172.21.15.64:0/1291273050 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-20T12:50:26.211 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:26.206+0000 7f3bb37fe700 1 -- 172.21.15.64:0/1291273050 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bb40ff0a0 con 0x7f3bb406ada0 2024-02-20T12:50:26.211 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:26.206+0000 7f3bb37fe700 1 --2- 172.21.15.64:0/1291273050 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3bb406ada0 0x7f3bb40fea70 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 rx=0x7f3ba400b5f0 tx=0x7f3ba4006ee0).ready entity=mon.0 client_cookie=f37ece68c64b3141 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:50:26.211 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:26.206+0000 7f3bb27fc700 1 -- 172.21.15.64:0/1291273050 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3ba400d600 con 0x7f3bb406ada0 2024-02-20T12:50:26.211 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:26.206+0000 7f3bb27fc700 1 -- 172.21.15.64:0/1291273050 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(25 keys) v1 ==== 997+0+0 (secure 0 0 0) 0x7f3ba400dbc0 con 0x7f3bb406ada0 2024-02-20T12:50:26.211 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:26.206+0000 7f3bb27fc700 1 -- 172.21.15.64:0/1291273050 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3ba4011b90 con 0x7f3bb406ada0 2024-02-20T12:50:26.211 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:26.206+0000 7f3bb8fec700 1 -- 172.21.15.64:0/1291273050 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3bb406ada0 msgr2=0x7f3bb40fea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:50:26.211 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:26.206+0000 7f3bb8fec700 1 --2- 172.21.15.64:0/1291273050 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3bb406ada0 0x7f3bb40fea70 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 rx=0x7f3ba400b5f0 tx=0x7f3ba4006ee0).stop 2024-02-20T12:50:26.211 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:26.206+0000 7f3bb8fec700 1 -- 172.21.15.64:0/1291273050 shutdown_connections 2024-02-20T12:50:26.211 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:26.206+0000 7f3bb8fec700 1 --2- 172.21.15.64:0/1291273050 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3bb406ada0 0x7f3bb40fea70 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:50:26.211 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:26.206+0000 7f3bb8fec700 1 -- 172.21.15.64:0/1291273050 >> 172.21.15.64:0/1291273050 conn(0x7f3bb40f64d0 msgr2=0x7f3bb40f88f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:50:26.211 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:26.207+0000 7f3bb8fec700 1 -- 172.21.15.64:0/1291273050 shutdown_connections 2024-02-20T12:50:26.211 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:26.207+0000 7f3bb8fec700 1 -- 172.21.15.64:0/1291273050 wait complete. 2024-02-20T12:50:26.211 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:26.207+0000 7f3bb8fec700 1 Processor -- start 2024-02-20T12:50:26.212 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:26.208+0000 7f3bb8fec700 1 -- start start 2024-02-20T12:50:26.212 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:26.208+0000 7f3bb8fec700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3bb406ada0 0x7f3bb410ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:50:26.212 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:26.208+0000 7f3bb8fec700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bb410b010 con 0x7f3bb406ada0 2024-02-20T12:50:26.212 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:26.208+0000 7f3bb37fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3bb406ada0 0x7f3bb410ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:50:26.212 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:26.208+0000 7f3bb37fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3bb406ada0 0x7f3bb410ab20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:49486/0 (socket says 172.21.15.64:49486) 2024-02-20T12:50:26.212 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:26.208+0000 7f3bb37fe700 1 -- 172.21.15.64:0/3169261973 learned_addr learned my addr 172.21.15.64:0/3169261973 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-20T12:50:26.212 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:26.208+0000 7f3bb37fe700 1 -- 172.21.15.64:0/3169261973 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ba400b200 con 0x7f3bb406ada0 2024-02-20T12:50:26.212 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:26.208+0000 7f3bb37fe700 1 --2- 172.21.15.64:0/3169261973 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3bb406ada0 0x7f3bb410ab20 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0x7f3ba4006800 tx=0x7f3ba4006ee0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:50:26.212 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:26.208+0000 7f3b9bfff700 1 -- 172.21.15.64:0/3169261973 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3ba400de50 con 0x7f3bb406ada0 2024-02-20T12:50:26.212 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:26.209+0000 7f3b9bfff700 1 -- 172.21.15.64:0/3169261973 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(25 keys) v1 ==== 997+0+0 (secure 0 0 0) 0x7f3ba401b870 con 0x7f3bb406ada0 2024-02-20T12:50:26.212 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:26.209+0000 7f3bb8fec700 1 -- 172.21.15.64:0/3169261973 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3bb410b210 con 0x7f3bb406ada0 2024-02-20T12:50:26.212 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:26.209+0000 7f3b9bfff700 1 -- 172.21.15.64:0/3169261973 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3ba401a7a0 con 0x7f3bb406ada0 2024-02-20T12:50:26.212 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:26.209+0000 7f3bb8fec700 1 -- 172.21.15.64:0/3169261973 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3bb410b6a0 con 0x7f3bb406ada0 2024-02-20T12:50:26.213 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:26.209+0000 7f3b9bfff700 1 -- 172.21.15.64:0/3169261973 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 8) v1 ==== 37095+0+0 (secure 0 0 0) 0x7f3ba400f070 con 0x7f3bb406ada0 2024-02-20T12:50:26.213 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:26.209+0000 7f3b9bfff700 1 --2- 172.21.15.64:0/3169261973 >> [v2:172.21.15.64:6800/1362908731,v1:172.21.15.64:6801/1362908731] conn(0x7f3ba0030ff0 0x7f3ba00334b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:50:26.214 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:26.209+0000 7f3b9bfff700 1 -- 172.21.15.64:0/3169261973 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 893+0+0 (secure 0 0 0) 0x7f3ba4043a10 con 0x7f3bb406ada0 2024-02-20T12:50:26.214 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:26.210+0000 7f3bb2ffd700 1 --2- 172.21.15.64:0/3169261973 >> [v2:172.21.15.64:6800/1362908731,v1:172.21.15.64:6801/1362908731] conn(0x7f3ba0030ff0 0x7f3ba00334b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:50:26.214 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:26.210+0000 7f3bb8fec700 1 -- 172.21.15.64:0/3169261973 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3bb4045bb0 con 0x7f3bb406ada0 2024-02-20T12:50:26.214 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:26.210+0000 7f3bb2ffd700 1 --2- 172.21.15.64:0/3169261973 >> [v2:172.21.15.64:6800/1362908731,v1:172.21.15.64:6801/1362908731] conn(0x7f3ba0030ff0 0x7f3ba00334b0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0x7f3ba80096a0 tx=0x7f3ba8006b40).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:50:26.215 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:26.214+0000 7f3b9bfff700 1 -- 172.21.15.64:0/3169261973 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+153237 (secure 0 0 0) 0x7f3ba4016030 con 0x7f3bb406ada0 2024-02-20T12:50:26.376 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:26.374+0000 7f3bb8fec700 1 -- 172.21.15.64:0/3169261973 --> [v2:172.21.15.64:6800/1362908731,v1:172.21.15.64:6801/1362908731] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) v1 -- 0x7f3bb4104b20 con 0x7f3ba0030ff0 2024-02-20T12:50:26.376 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:26.375+0000 7f3b9bfff700 1 -- 172.21.15.64:0/3169261973 <== mgr.14116 v2:172.21.15.64:6800/1362908731 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+16 (secure 0 0 0) 0x7f3bb4104b20 con 0x7f3ba0030ff0 2024-02-20T12:50:26.376 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: value unchanged 2024-02-20T12:50:26.378 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:26.377+0000 7f3bb8fec700 1 -- 172.21.15.64:0/3169261973 >> [v2:172.21.15.64:6800/1362908731,v1:172.21.15.64:6801/1362908731] conn(0x7f3ba0030ff0 msgr2=0x7f3ba00334b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:50:26.379 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:26.377+0000 7f3bb8fec700 1 --2- 172.21.15.64:0/3169261973 >> [v2:172.21.15.64:6800/1362908731,v1:172.21.15.64:6801/1362908731] conn(0x7f3ba0030ff0 0x7f3ba00334b0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0x7f3ba80096a0 tx=0x7f3ba8006b40).stop 2024-02-20T12:50:26.379 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:26.377+0000 7f3bb8fec700 1 -- 172.21.15.64:0/3169261973 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3bb406ada0 msgr2=0x7f3bb410ab20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:50:26.379 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:26.377+0000 7f3bb8fec700 1 --2- 172.21.15.64:0/3169261973 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3bb406ada0 0x7f3bb410ab20 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0x7f3ba4006800 tx=0x7f3ba4006ee0).stop 2024-02-20T12:50:26.379 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:26.377+0000 7f3bb8fec700 1 -- 172.21.15.64:0/3169261973 shutdown_connections 2024-02-20T12:50:26.379 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:26.377+0000 7f3bb8fec700 1 --2- 172.21.15.64:0/3169261973 >> [v2:172.21.15.64:6800/1362908731,v1:172.21.15.64:6801/1362908731] conn(0x7f3ba0030ff0 0x7f3ba00334b0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:50:26.379 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:26.377+0000 7f3bb8fec700 1 --2- 172.21.15.64:0/3169261973 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3bb406ada0 0x7f3bb410ab20 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:50:26.379 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:26.377+0000 7f3bb8fec700 1 -- 172.21.15.64:0/3169261973 >> 172.21.15.64:0/3169261973 conn(0x7f3bb40f64d0 msgr2=0x7f3bb40f7ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:50:26.379 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:26.378+0000 7f3bb8fec700 1 -- 172.21.15.64:0/3169261973 shutdown_connections 2024-02-20T12:50:26.379 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:26.378+0000 7f3bb8fec700 1 -- 172.21.15.64:0/3169261973 wait complete. 2024-02-20T12:50:26.998 INFO:teuthology.orchestra.run.smithi064.stderr:Generating ssh key... 2024-02-20T12:50:27.272 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:27 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:50:26.376245+0000 mgr.smithi064.vriyor (mgr.14116) 4 : audit [DBG] from='client.14128 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-02-20T12:50:27.553 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:27.519+0000 7f97bbe07700 1 Processor -- start 2024-02-20T12:50:27.553 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:27.520+0000 7f97bbe07700 1 -- start start 2024-02-20T12:50:27.553 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:27.520+0000 7f97bbe07700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f97b4027610 0x7f97b4101040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-02-20T12:50:27.553 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:27.520+0000 7f97bbe07700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97b40fca30 con 0x7f97b4027610 2024-02-20T12:50:27.553 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:27.520+0000 7f97b9ba3700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f97b4027610 0x7f97b4101040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:50:27.553 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:27.520+0000 7f97b9ba3700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f97b4027610 0x7f97b4101040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:49490/0 (socket says 172.21.15.64:49490) 2024-02-20T12:50:27.553 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:27.520+0000 7f97b9ba3700 1 -- 172.21.15.64:0/926667814 learned_addr learned my addr 172.21.15.64:0/926667814 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-20T12:50:27.554 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:27.520+0000 7f97b9ba3700 1 -- 172.21.15.64:0/926667814 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97b4101530 con 0x7f97b4027610 2024-02-20T12:50:27.554 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:27.520+0000 7f97b9ba3700 1 --2- 172.21.15.64:0/926667814 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f97b4027610 0x7f97b4101040 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7f97a800b5f0 tx=0x7f97a8006ee0).ready entity=mon.0 client_cookie=62deea1ba63d6c1c server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:50:27.554 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:27.520+0000 7f97b8ba1700 1 -- 172.21.15.64:0/926667814 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f97a800d600 con 0x7f97b4027610 2024-02-20T12:50:27.554 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:27.520+0000 7f97b8ba1700 1 -- 172.21.15.64:0/926667814 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(25 keys) v1 ==== 997+0+0 (secure 0 0 0) 0x7f97a800dbc0 con 0x7f97b4027610 2024-02-20T12:50:27.554 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:27.520+0000 7f97b8ba1700 1 -- 172.21.15.64:0/926667814 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f97a8011b90 con 0x7f97b4027610 2024-02-20T12:50:27.554 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:27.521+0000 7f97bbe07700 1 -- 172.21.15.64:0/926667814 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f97b4027610 msgr2=0x7f97b4101040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:50:27.554 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:27.521+0000 7f97bbe07700 1 --2- 172.21.15.64:0/926667814 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f97b4027610 0x7f97b4101040 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7f97a800b5f0 tx=0x7f97a8006ee0).stop 2024-02-20T12:50:27.554 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:27.521+0000 7f97bbe07700 1 -- 172.21.15.64:0/926667814 shutdown_connections 2024-02-20T12:50:27.554 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:27.521+0000 7f97bbe07700 1 --2- 172.21.15.64:0/926667814 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f97b4027610 0x7f97b4101040 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:50:27.554 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:27.521+0000 7f97bbe07700 1 -- 172.21.15.64:0/926667814 >> 172.21.15.64:0/926667814 conn(0x7f97b40f6680 msgr2=0x7f97b40f8aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:50:27.554 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:27.521+0000 7f97bbe07700 1 -- 172.21.15.64:0/926667814 shutdown_connections 2024-02-20T12:50:27.554 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:27.521+0000 7f97bbe07700 1 -- 172.21.15.64:0/926667814 wait complete. 2024-02-20T12:50:27.554 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:27.521+0000 7f97bbe07700 1 Processor -- start 2024-02-20T12:50:27.555 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:27.522+0000 7f97bbe07700 1 -- start start 2024-02-20T12:50:27.555 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:27.522+0000 7f97bbe07700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f97b4027610 0x7f97b410cec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:50:27.555 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:27.522+0000 7f97bbe07700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97b410d3b0 con 0x7f97b4027610 2024-02-20T12:50:27.555 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:27.522+0000 7f97b9ba3700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f97b4027610 0x7f97b410cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:50:27.555 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:27.522+0000 7f97b9ba3700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f97b4027610 0x7f97b410cec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:49502/0 (socket says 172.21.15.64:49502) 2024-02-20T12:50:27.555 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:27.522+0000 7f97b9ba3700 1 -- 172.21.15.64:0/2882713188 learned_addr learned my addr 172.21.15.64:0/2882713188 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-20T12:50:27.555 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:27.522+0000 7f97b9ba3700 1 -- 172.21.15.64:0/2882713188 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97a800b200 con 0x7f97b4027610 2024-02-20T12:50:27.555 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:27.522+0000 7f97b9ba3700 1 --2- 172.21.15.64:0/2882713188 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f97b4027610 0x7f97b410cec0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7f97a8006800 tx=0x7f97a8006ee0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:50:27.555 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:27.522+0000 7f97a67fc700 1 -- 172.21.15.64:0/2882713188 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f97a800de50 con 0x7f97b4027610 2024-02-20T12:50:27.555 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:27.522+0000 7f97a67fc700 1 -- 172.21.15.64:0/2882713188 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(25 keys) v1 ==== 997+0+0 (secure 0 0 0) 0x7f97a801b870 con 0x7f97b4027610 2024-02-20T12:50:27.555 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:27.522+0000 7f97bbe07700 1 -- 172.21.15.64:0/2882713188 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97b410d5b0 con 0x7f97b4027610 2024-02-20T12:50:27.555 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:27.522+0000 7f97a67fc700 1 -- 172.21.15.64:0/2882713188 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f97a801a7a0 con 0x7f97b4027610 2024-02-20T12:50:27.556 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:27.522+0000 7f97bbe07700 1 -- 172.21.15.64:0/2882713188 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97b410da40 con 0x7f97b4027610 2024-02-20T12:50:27.556 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:27.523+0000 7f97a67fc700 1 -- 172.21.15.64:0/2882713188 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 8) v1 ==== 37095+0+0 (secure 0 0 0) 0x7f97a800f070 con 0x7f97b4027610 2024-02-20T12:50:27.556 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:27.523+0000 7f97a67fc700 1 --2- 172.21.15.64:0/2882713188 >> [v2:172.21.15.64:6800/1362908731,v1:172.21.15.64:6801/1362908731] conn(0x7f979c030ff0 0x7f979c0334b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:50:27.556 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:27.523+0000 7f97a67fc700 1 -- 172.21.15.64:0/2882713188 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 893+0+0 (secure 0 0 0) 0x7f97a8042b80 con 0x7f97b4027610 2024-02-20T12:50:27.556 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:27.523+0000 7f97b93a2700 1 --2- 172.21.15.64:0/2882713188 >> [v2:172.21.15.64:6800/1362908731,v1:172.21.15.64:6801/1362908731] conn(0x7f979c030ff0 0x7f979c0334b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:50:27.556 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:27.524+0000 7f97b93a2700 1 --2- 172.21.15.64:0/2882713188 >> [v2:172.21.15.64:6800/1362908731,v1:172.21.15.64:6801/1362908731] conn(0x7f979c030ff0 0x7f979c0334b0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0x7f97b00096a0 tx=0x7f97b0006b40).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:50:27.556 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:27.524+0000 7f97bbe07700 1 -- 172.21.15.64:0/2882713188 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97b4045bb0 con 0x7f97b4027610 2024-02-20T12:50:27.556 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:27.528+0000 7f97a67fc700 1 -- 172.21.15.64:0/2882713188 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+153237 (secure 0 0 0) 0x7f97a8016030 con 0x7f97b4027610 2024-02-20T12:50:27.692 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:27.690+0000 7f97bbe07700 1 -- 172.21.15.64:0/2882713188 --> [v2:172.21.15.64:6800/1362908731,v1:172.21.15.64:6801/1362908731] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) v1 -- 0x7f97b40008d0 con 0x7f979c030ff0 2024-02-20T12:50:27.962 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:27.960+0000 7f97a67fc700 1 -- 172.21.15.64:0/2882713188 <== mgr.14116 v2:172.21.15.64:6800/1362908731 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f97b40008d0 con 0x7f979c030ff0 2024-02-20T12:50:27.964 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:27.962+0000 7f97bbe07700 1 -- 172.21.15.64:0/2882713188 >> [v2:172.21.15.64:6800/1362908731,v1:172.21.15.64:6801/1362908731] conn(0x7f979c030ff0 msgr2=0x7f979c0334b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:50:27.964 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:27.962+0000 7f97bbe07700 1 --2- 172.21.15.64:0/2882713188 >> [v2:172.21.15.64:6800/1362908731,v1:172.21.15.64:6801/1362908731] conn(0x7f979c030ff0 0x7f979c0334b0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0x7f97b00096a0 tx=0x7f97b0006b40).stop 2024-02-20T12:50:27.964 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:27.962+0000 7f97bbe07700 1 -- 172.21.15.64:0/2882713188 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f97b4027610 msgr2=0x7f97b410cec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:50:27.964 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:27.962+0000 7f97bbe07700 1 --2- 172.21.15.64:0/2882713188 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f97b4027610 0x7f97b410cec0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7f97a8006800 tx=0x7f97a8006ee0).stop 2024-02-20T12:50:27.964 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:27.962+0000 7f97bbe07700 1 -- 172.21.15.64:0/2882713188 shutdown_connections 2024-02-20T12:50:27.964 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:27.962+0000 7f97bbe07700 1 --2- 172.21.15.64:0/2882713188 >> [v2:172.21.15.64:6800/1362908731,v1:172.21.15.64:6801/1362908731] conn(0x7f979c030ff0 0x7f979c0334b0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:50:27.964 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:27.962+0000 7f97bbe07700 1 --2- 172.21.15.64:0/2882713188 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f97b4027610 0x7f97b410cec0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:50:27.964 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:27.962+0000 7f97bbe07700 1 -- 172.21.15.64:0/2882713188 >> 172.21.15.64:0/2882713188 conn(0x7f97b40f6680 msgr2=0x7f97b40f8110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:50:27.965 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:27.962+0000 7f97bbe07700 1 -- 172.21.15.64:0/2882713188 shutdown_connections 2024-02-20T12:50:27.965 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:27.962+0000 7f97bbe07700 1 -- 172.21.15.64:0/2882713188 wait complete. 2024-02-20T12:50:29.293 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:29.253+0000 7f9fe9662700 1 Processor -- start 2024-02-20T12:50:29.293 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:29.253+0000 7f9fe9662700 1 -- start start 2024-02-20T12:50:29.293 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:29.253+0000 7f9fe9662700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9fe40fad70 0x7f9fe40fb190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-02-20T12:50:29.293 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:29.253+0000 7f9fe9662700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fe40fb680 con 0x7f9fe40fad70 2024-02-20T12:50:29.293 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:29.253+0000 7f9fe3fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9fe40fad70 0x7f9fe40fb190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:50:29.293 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:29.253+0000 7f9fe3fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9fe40fad70 0x7f9fe40fb190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:49504/0 (socket says 172.21.15.64:49504) 2024-02-20T12:50:29.294 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:29.253+0000 7f9fe3fff700 1 -- 172.21.15.64:0/4161475499 learned_addr learned my addr 172.21.15.64:0/4161475499 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-20T12:50:29.294 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:29.254+0000 7f9fe3fff700 1 -- 172.21.15.64:0/4161475499 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fe40fb7c0 con 0x7f9fe40fad70 2024-02-20T12:50:29.294 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:29.254+0000 7f9fe3fff700 1 --2- 172.21.15.64:0/4161475499 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9fe40fad70 0x7f9fe40fb190 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7f9fd400b5f0 tx=0x7f9fd4006ee0).ready entity=mon.0 client_cookie=de7acc0625e9c214 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:50:29.294 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:29.254+0000 7f9fe2ffd700 1 -- 172.21.15.64:0/4161475499 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9fd400d600 con 0x7f9fe40fad70 2024-02-20T12:50:29.294 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:29.254+0000 7f9fe2ffd700 1 -- 172.21.15.64:0/4161475499 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(25 keys) v1 ==== 997+0+0 (secure 0 0 0) 0x7f9fd400dbc0 con 0x7f9fe40fad70 2024-02-20T12:50:29.294 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:29.254+0000 7f9fe2ffd700 1 -- 172.21.15.64:0/4161475499 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9fd4011b90 con 0x7f9fe40fad70 2024-02-20T12:50:29.294 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:29.254+0000 7f9fe9662700 1 -- 172.21.15.64:0/4161475499 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9fe40fad70 msgr2=0x7f9fe40fb190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:50:29.294 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:29.254+0000 7f9fe9662700 1 --2- 172.21.15.64:0/4161475499 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9fe40fad70 0x7f9fe40fb190 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7f9fd400b5f0 tx=0x7f9fd4006ee0).stop 2024-02-20T12:50:29.294 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:29.255+0000 7f9fe9662700 1 -- 172.21.15.64:0/4161475499 shutdown_connections 2024-02-20T12:50:29.294 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:29.255+0000 7f9fe9662700 1 --2- 172.21.15.64:0/4161475499 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9fe40fad70 0x7f9fe40fb190 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:50:29.294 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:29.255+0000 7f9fe9662700 1 -- 172.21.15.64:0/4161475499 >> 172.21.15.64:0/4161475499 conn(0x7f9fe40f64d0 msgr2=0x7f9fe40f88f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:50:29.294 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:29.255+0000 7f9fe9662700 1 -- 172.21.15.64:0/4161475499 shutdown_connections 2024-02-20T12:50:29.294 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:29.255+0000 7f9fe9662700 1 -- 172.21.15.64:0/4161475499 wait complete. 2024-02-20T12:50:29.295 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:29.255+0000 7f9fe9662700 1 Processor -- start 2024-02-20T12:50:29.295 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:29.255+0000 7f9fe9662700 1 -- start start 2024-02-20T12:50:29.295 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:29.256+0000 7f9fe9662700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9fe40fad70 0x7f9fe410d820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:50:29.295 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:29.256+0000 7f9fe9662700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fe406ad60 con 0x7f9fe40fad70 2024-02-20T12:50:29.295 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:29.256+0000 7f9fe3fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9fe40fad70 0x7f9fe410d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:50:29.295 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:29.256+0000 7f9fe3fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9fe40fad70 0x7f9fe410d820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:49506/0 (socket says 172.21.15.64:49506) 2024-02-20T12:50:29.295 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:29.256+0000 7f9fe3fff700 1 -- 172.21.15.64:0/2590317092 learned_addr learned my addr 172.21.15.64:0/2590317092 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-20T12:50:29.295 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:29.256+0000 7f9fe3fff700 1 -- 172.21.15.64:0/2590317092 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fd400b200 con 0x7f9fe40fad70 2024-02-20T12:50:29.295 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:29.256+0000 7f9fe3fff700 1 --2- 172.21.15.64:0/2590317092 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9fe40fad70 0x7f9fe410d820 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7f9fd4000c00 tx=0x7f9fd4006ee0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:50:29.295 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:29.256+0000 7f9fe0ff9700 1 -- 172.21.15.64:0/2590317092 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9fd401b410 con 0x7f9fe40fad70 2024-02-20T12:50:29.295 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:29.256+0000 7f9fe0ff9700 1 -- 172.21.15.64:0/2590317092 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(25 keys) v1 ==== 997+0+0 (secure 0 0 0) 0x7f9fd401b9d0 con 0x7f9fe40fad70 2024-02-20T12:50:29.295 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:29.256+0000 7f9fe9662700 1 -- 172.21.15.64:0/2590317092 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9fe406af60 con 0x7f9fe40fad70 2024-02-20T12:50:29.295 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:29.256+0000 7f9fe0ff9700 1 -- 172.21.15.64:0/2590317092 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9fd401a600 con 0x7f9fe40fad70 2024-02-20T12:50:29.296 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:29.256+0000 7f9fe9662700 1 -- 172.21.15.64:0/2590317092 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9fe410ded0 con 0x7f9fe40fad70 2024-02-20T12:50:29.296 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:29.257+0000 7f9fe0ff9700 1 -- 172.21.15.64:0/2590317092 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 8) v1 ==== 37095+0+0 (secure 0 0 0) 0x7f9fd402b070 con 0x7f9fe40fad70 2024-02-20T12:50:29.296 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:29.257+0000 7f9fe0ff9700 1 --2- 172.21.15.64:0/2590317092 >> [v2:172.21.15.64:6800/1362908731,v1:172.21.15.64:6801/1362908731] conn(0x7f9fc4030ff0 0x7f9fc40334b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:50:29.296 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:29.257+0000 7f9fe0ff9700 1 -- 172.21.15.64:0/2590317092 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 893+0+0 (secure 0 0 0) 0x7f9fd4042b70 con 0x7f9fe40fad70 2024-02-20T12:50:29.296 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:29.257+0000 7f9fe37fe700 1 --2- 172.21.15.64:0/2590317092 >> [v2:172.21.15.64:6800/1362908731,v1:172.21.15.64:6801/1362908731] conn(0x7f9fc4030ff0 0x7f9fc40334b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:50:29.296 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:29.257+0000 7f9fe9662700 1 -- 172.21.15.64:0/2590317092 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9fe4045bb0 con 0x7f9fe40fad70 2024-02-20T12:50:29.296 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:29.258+0000 7f9fe37fe700 1 --2- 172.21.15.64:0/2590317092 >> [v2:172.21.15.64:6800/1362908731,v1:172.21.15.64:6801/1362908731] conn(0x7f9fc4030ff0 0x7f9fc40334b0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 rx=0x7f9fcc0096a0 tx=0x7f9fcc006b40).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:50:29.296 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:29.262+0000 7f9fe0ff9700 1 -- 172.21.15.64:0/2590317092 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+153237 (secure 0 0 0) 0x7f9fd4016020 con 0x7f9fe40fad70 2024-02-20T12:50:29.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:28 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cephadm 2024-02-20T12:50:27.692531+0000 mgr.smithi064.vriyor (mgr.14116) 6 : cephadm [INF] Generating ssh key... 2024-02-20T12:50:29.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:28 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:50:27.959242+0000 mon.smithi064 (mon.0) 53 : audit [INF] from='mgr.14116 172.21.15.64:0/330181513' entity='mgr.smithi064.vriyor' 2024-02-20T12:50:29.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:28 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:50:27.960925+0000 mon.smithi064 (mon.0) 54 : audit [INF] from='mgr.14116 172.21.15.64:0/330181513' entity='mgr.smithi064.vriyor' 2024-02-20T12:50:29.425 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:29.424+0000 7f9fe9662700 1 -- 172.21.15.64:0/2590317092 --> [v2:172.21.15.64:6800/1362908731,v1:172.21.15.64:6801/1362908731] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) v1 -- 0x7f9fe40fcb90 con 0x7f9fc4030ff0 2024-02-20T12:50:29.426 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:29.425+0000 7f9fe0ff9700 1 -- 172.21.15.64:0/2590317092 <== mgr.14116 v2:172.21.15.64:6800/1362908731 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+595 (secure 0 0 0) 0x7f9fe40fcb90 con 0x7f9fc4030ff0 2024-02-20T12:50:29.426 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDKOQCSumDj3118UsALF+LqZu5FOGmhIFdUrATcTf8tvctNd0xnOBj3cDTxFUzyVjPw599qypes3HlfCkN+PUvf6tXuoEOigfi07scE0UkkxmCd9d5+cXmXw+ugH+0e75mpJaX04hcG02wrHx5YjHpKz6K+XgWpW+xizjFJO2/g4ceNUBfW93qKexpstyhP6CwGm7ywxreZcFW+QUiEWVXgPYrzMvDUsdqN+5P6KjVvEGq5jECJEstco/r1MZgOHCFAr3v7gt7BYav0pFg1aQx6+NHExOMtq07OA15dD3MB0wk+k0ypAqKM3eV3M2nV/kK+UXjuhAelnDe1LTAD/P/opKtmBUSzh+S14L6VarVdF3LaX1+THnOMNHkmR6BoU+BiQ1hz38p/lDBlv2/0HLbZbbLSnV8Xzl481F90qiNBYblm+wbFFxCXi6woVETn+J5yikpC3rHKOa0RnDIouxA4V698sI42MY77qWmwWE8OvPYnGsXkuNJkCacCMTzNd0s= ceph-680ee126-cfee-11ee-95bb-87774f69a715 2024-02-20T12:50:29.428 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:29.426+0000 7f9fe9662700 1 -- 172.21.15.64:0/2590317092 >> [v2:172.21.15.64:6800/1362908731,v1:172.21.15.64:6801/1362908731] conn(0x7f9fc4030ff0 msgr2=0x7f9fc40334b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:50:29.428 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:29.426+0000 7f9fe9662700 1 --2- 172.21.15.64:0/2590317092 >> [v2:172.21.15.64:6800/1362908731,v1:172.21.15.64:6801/1362908731] conn(0x7f9fc4030ff0 0x7f9fc40334b0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 rx=0x7f9fcc0096a0 tx=0x7f9fcc006b40).stop 2024-02-20T12:50:29.428 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:29.426+0000 7f9fe9662700 1 -- 172.21.15.64:0/2590317092 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9fe40fad70 msgr2=0x7f9fe410d820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:50:29.428 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:29.426+0000 7f9fe9662700 1 --2- 172.21.15.64:0/2590317092 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9fe40fad70 0x7f9fe410d820 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7f9fd4000c00 tx=0x7f9fd4006ee0).stop 2024-02-20T12:50:29.429 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:29.426+0000 7f9fe9662700 1 -- 172.21.15.64:0/2590317092 shutdown_connections 2024-02-20T12:50:29.429 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:29.426+0000 7f9fe9662700 1 --2- 172.21.15.64:0/2590317092 >> [v2:172.21.15.64:6800/1362908731,v1:172.21.15.64:6801/1362908731] conn(0x7f9fc4030ff0 0x7f9fc40334b0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:50:29.429 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:29.426+0000 7f9fe9662700 1 --2- 172.21.15.64:0/2590317092 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9fe40fad70 0x7f9fe410d820 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:50:29.429 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:29.426+0000 7f9fe9662700 1 -- 172.21.15.64:0/2590317092 >> 172.21.15.64:0/2590317092 conn(0x7f9fe40f64d0 msgr2=0x7f9fe40f7f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:50:29.429 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:29.427+0000 7f9fe9662700 1 -- 172.21.15.64:0/2590317092 shutdown_connections 2024-02-20T12:50:29.429 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:29.427+0000 7f9fe9662700 1 -- 172.21.15.64:0/2590317092 wait complete. 2024-02-20T12:50:30.020 INFO:teuthology.orchestra.run.smithi064.stderr:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-02-20T12:50:30.020 INFO:teuthology.orchestra.run.smithi064.stderr:Adding key to root@localhost authorized_keys... 2024-02-20T12:50:30.021 INFO:teuthology.orchestra.run.smithi064.stderr:Adding host smithi064... 2024-02-20T12:50:30.287 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:30 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:50:29.426090+0000 mgr.smithi064.vriyor (mgr.14116) 7 : audit [DBG] from='client.14132 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-02-20T12:50:30.639 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:30.634+0000 7f12c9bf6700 1 Processor -- start 2024-02-20T12:50:30.639 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:30.634+0000 7f12c9bf6700 1 -- start start 2024-02-20T12:50:30.639 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:30.634+0000 7f12c9bf6700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f12c406ad60 0x7f12c4101060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-02-20T12:50:30.639 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:30.634+0000 7f12c9bf6700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12c40fca50 con 0x7f12c406ad60 2024-02-20T12:50:30.640 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:30.635+0000 7f12c37fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f12c406ad60 0x7f12c4101060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:50:30.640 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:30.635+0000 7f12c37fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f12c406ad60 0x7f12c4101060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:49518/0 (socket says 172.21.15.64:49518) 2024-02-20T12:50:30.640 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:30.635+0000 7f12c37fe700 1 -- 172.21.15.64:0/2119046092 learned_addr learned my addr 172.21.15.64:0/2119046092 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-20T12:50:30.640 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:30.635+0000 7f12c37fe700 1 -- 172.21.15.64:0/2119046092 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12c4101550 con 0x7f12c406ad60 2024-02-20T12:50:30.640 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:30.635+0000 7f12c37fe700 1 --2- 172.21.15.64:0/2119046092 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f12c406ad60 0x7f12c4101060 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7f12ac00b5f0 tx=0x7f12ac006ee0).ready entity=mon.0 client_cookie=d2fbe365e2305664 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:50:30.640 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:30.635+0000 7f12c27fc700 1 -- 172.21.15.64:0/2119046092 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f12ac00d600 con 0x7f12c406ad60 2024-02-20T12:50:30.640 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:30.635+0000 7f12c27fc700 1 -- 172.21.15.64:0/2119046092 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(25 keys) v1 ==== 997+0+0 (secure 0 0 0) 0x7f12ac00dbc0 con 0x7f12c406ad60 2024-02-20T12:50:30.640 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:30.635+0000 7f12c27fc700 1 -- 172.21.15.64:0/2119046092 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f12ac011b90 con 0x7f12c406ad60 2024-02-20T12:50:30.640 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:30.636+0000 7f12c9bf6700 1 -- 172.21.15.64:0/2119046092 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f12c406ad60 msgr2=0x7f12c4101060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:50:30.640 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:30.636+0000 7f12c9bf6700 1 --2- 172.21.15.64:0/2119046092 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f12c406ad60 0x7f12c4101060 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7f12ac00b5f0 tx=0x7f12ac006ee0).stop 2024-02-20T12:50:30.640 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:30.636+0000 7f12c9bf6700 1 -- 172.21.15.64:0/2119046092 shutdown_connections 2024-02-20T12:50:30.640 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:30.636+0000 7f12c9bf6700 1 --2- 172.21.15.64:0/2119046092 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f12c406ad60 0x7f12c4101060 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:50:30.640 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:30.636+0000 7f12c9bf6700 1 -- 172.21.15.64:0/2119046092 >> 172.21.15.64:0/2119046092 conn(0x7f12c40f6680 msgr2=0x7f12c40f8aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:50:30.640 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:30.636+0000 7f12c9bf6700 1 -- 172.21.15.64:0/2119046092 shutdown_connections 2024-02-20T12:50:30.641 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:30.636+0000 7f12c9bf6700 1 -- 172.21.15.64:0/2119046092 wait complete. 2024-02-20T12:50:30.641 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:30.636+0000 7f12c9bf6700 1 Processor -- start 2024-02-20T12:50:30.641 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:30.637+0000 7f12c9bf6700 1 -- start start 2024-02-20T12:50:30.641 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:30.637+0000 7f12c9bf6700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f12c406ad60 0x7f12c410cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:50:30.641 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:30.637+0000 7f12c9bf6700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12c410d3f0 con 0x7f12c406ad60 2024-02-20T12:50:30.641 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:30.637+0000 7f12c37fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f12c406ad60 0x7f12c410cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:50:30.641 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:30.637+0000 7f12c37fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f12c406ad60 0x7f12c410cf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:49528/0 (socket says 172.21.15.64:49528) 2024-02-20T12:50:30.641 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:30.637+0000 7f12c37fe700 1 -- 172.21.15.64:0/1198626792 learned_addr learned my addr 172.21.15.64:0/1198626792 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-20T12:50:30.641 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:30.637+0000 7f12c37fe700 1 -- 172.21.15.64:0/1198626792 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12ac00b200 con 0x7f12c406ad60 2024-02-20T12:50:30.641 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:30.637+0000 7f12c37fe700 1 --2- 172.21.15.64:0/1198626792 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f12c406ad60 0x7f12c410cf00 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7f12ac000c00 tx=0x7f12ac006ee0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:50:30.641 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:30.638+0000 7f12bbfff700 1 -- 172.21.15.64:0/1198626792 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f12ac00de00 con 0x7f12c406ad60 2024-02-20T12:50:30.641 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:30.638+0000 7f12c9bf6700 1 -- 172.21.15.64:0/1198626792 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12c410d5f0 con 0x7f12c406ad60 2024-02-20T12:50:30.641 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:30.638+0000 7f12c9bf6700 1 -- 172.21.15.64:0/1198626792 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12c410da80 con 0x7f12c406ad60 2024-02-20T12:50:30.641 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:30.638+0000 7f12bbfff700 1 -- 172.21.15.64:0/1198626792 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(25 keys) v1 ==== 997+0+0 (secure 0 0 0) 0x7f12ac01b870 con 0x7f12c406ad60 2024-02-20T12:50:30.641 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:30.638+0000 7f12bbfff700 1 -- 172.21.15.64:0/1198626792 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f12ac01a550 con 0x7f12c406ad60 2024-02-20T12:50:30.642 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:30.639+0000 7f12bbfff700 1 -- 172.21.15.64:0/1198626792 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 8) v1 ==== 37095+0+0 (secure 0 0 0) 0x7f12ac02b070 con 0x7f12c406ad60 2024-02-20T12:50:30.642 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:30.639+0000 7f12c9bf6700 1 -- 172.21.15.64:0/1198626792 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12a8004fa0 con 0x7f12c406ad60 2024-02-20T12:50:30.643 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:30.639+0000 7f12bbfff700 1 --2- 172.21.15.64:0/1198626792 >> [v2:172.21.15.64:6800/1362908731,v1:172.21.15.64:6801/1362908731] conn(0x7f12a4030fa0 0x7f12a4033460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:50:30.643 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:30.639+0000 7f12bbfff700 1 -- 172.21.15.64:0/1198626792 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 893+0+0 (secure 0 0 0) 0x7f12ac043620 con 0x7f12c406ad60 2024-02-20T12:50:30.643 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:30.639+0000 7f12c2ffd700 1 --2- 172.21.15.64:0/1198626792 >> [v2:172.21.15.64:6800/1362908731,v1:172.21.15.64:6801/1362908731] conn(0x7f12a4030fa0 0x7f12a4033460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:50:30.643 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:30.639+0000 7f12c2ffd700 1 --2- 172.21.15.64:0/1198626792 >> [v2:172.21.15.64:6800/1362908731,v1:172.21.15.64:6801/1362908731] conn(0x7f12a4030fa0 0x7f12a4033460 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 rx=0x7f12b40096a0 tx=0x7f12b4006b40).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:50:30.644 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:30.643+0000 7f12bbfff700 1 -- 172.21.15.64:0/1198626792 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+153237 (secure 0 0 0) 0x7f12ac02b8f0 con 0x7f12c406ad60 2024-02-20T12:50:30.807 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:30.805+0000 7f12c9bf6700 1 -- 172.21.15.64:0/1198626792 --> [v2:172.21.15.64:6800/1362908731,v1:172.21.15.64:6801/1362908731] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi064", "addr": "172.21.15.64", "target": ["mon-mgr", ""]}) v1 -- 0x7f12a80029f0 con 0x7f12a4030fa0 2024-02-20T12:50:31.428 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: Added host 'smithi064' with addr '172.21.15.64' 2024-02-20T12:50:31.428 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:31.423+0000 7f12bbfff700 1 -- 172.21.15.64:0/1198626792 <== mgr.14116 v2:172.21.15.64:6800/1362908731 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+48 (secure 0 0 0) 0x7f12a80029f0 con 0x7f12a4030fa0 2024-02-20T12:50:31.429 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:31.426+0000 7f12c9bf6700 1 -- 172.21.15.64:0/1198626792 >> [v2:172.21.15.64:6800/1362908731,v1:172.21.15.64:6801/1362908731] conn(0x7f12a4030fa0 msgr2=0x7f12a4033460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:50:31.429 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:31.426+0000 7f12c9bf6700 1 --2- 172.21.15.64:0/1198626792 >> [v2:172.21.15.64:6800/1362908731,v1:172.21.15.64:6801/1362908731] conn(0x7f12a4030fa0 0x7f12a4033460 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 rx=0x7f12b40096a0 tx=0x7f12b4006b40).stop 2024-02-20T12:50:31.429 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:31.426+0000 7f12c9bf6700 1 -- 172.21.15.64:0/1198626792 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f12c406ad60 msgr2=0x7f12c410cf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:50:31.429 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:31.426+0000 7f12c9bf6700 1 --2- 172.21.15.64:0/1198626792 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f12c406ad60 0x7f12c410cf00 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7f12ac000c00 tx=0x7f12ac006ee0).stop 2024-02-20T12:50:31.429 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:31.427+0000 7f12c9bf6700 1 -- 172.21.15.64:0/1198626792 shutdown_connections 2024-02-20T12:50:31.429 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:31.427+0000 7f12c9bf6700 1 --2- 172.21.15.64:0/1198626792 >> [v2:172.21.15.64:6800/1362908731,v1:172.21.15.64:6801/1362908731] conn(0x7f12a4030fa0 0x7f12a4033460 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:50:31.429 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:31.427+0000 7f12c9bf6700 1 --2- 172.21.15.64:0/1198626792 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f12c406ad60 0x7f12c410cf00 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:50:31.429 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:31.427+0000 7f12c9bf6700 1 -- 172.21.15.64:0/1198626792 >> 172.21.15.64:0/1198626792 conn(0x7f12c40f6680 msgr2=0x7f12c40f80e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:50:31.429 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:31.427+0000 7f12c9bf6700 1 -- 172.21.15.64:0/1198626792 shutdown_connections 2024-02-20T12:50:31.429 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:31.427+0000 7f12c9bf6700 1 -- 172.21.15.64:0/1198626792 wait complete. 2024-02-20T12:50:31.453 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:31 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:50:30.807589+0000 mgr.smithi064.vriyor (mgr.14116) 8 : audit [DBG] from='client.14134 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi064", "addr": "172.21.15.64", "target": ["mon-mgr", ""]}]: dispatch 2024-02-20T12:50:32.118 INFO:teuthology.orchestra.run.smithi064.stderr:Deploying mon service with default placement... 2024-02-20T12:50:32.727 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:32.724+0000 7f04131e6700 1 Processor -- start 2024-02-20T12:50:32.728 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:32.724+0000 7f04131e6700 1 -- start start 2024-02-20T12:50:32.728 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:32.724+0000 7f04131e6700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f040c0faed0 0x7f040c0fb2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-02-20T12:50:32.728 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:32.724+0000 7f04131e6700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f040c0fb7e0 con 0x7f040c0faed0 2024-02-20T12:50:32.728 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:32.724+0000 7f0410f82700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f040c0faed0 0x7f040c0fb2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:50:32.728 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:32.724+0000 7f0410f82700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f040c0faed0 0x7f040c0fb2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57048/0 (socket says 172.21.15.64:57048) 2024-02-20T12:50:32.728 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:32.724+0000 7f0410f82700 1 -- 172.21.15.64:0/1013837196 learned_addr learned my addr 172.21.15.64:0/1013837196 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-20T12:50:32.728 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:32.724+0000 7f0410f82700 1 -- 172.21.15.64:0/1013837196 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f040c0fb920 con 0x7f040c0faed0 2024-02-20T12:50:32.728 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:32.724+0000 7f0410f82700 1 --2- 172.21.15.64:0/1013837196 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f040c0faed0 0x7f040c0fb2f0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7f040000b5f0 tx=0x7f0400006ee0).ready entity=mon.0 client_cookie=f660fa1a5807aa05 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:50:32.728 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:32.725+0000 7f040b7fe700 1 -- 172.21.15.64:0/1013837196 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f040000d600 con 0x7f040c0faed0 2024-02-20T12:50:32.728 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:32.725+0000 7f040b7fe700 1 -- 172.21.15.64:0/1013837196 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(25 keys) v1 ==== 997+0+0 (secure 0 0 0) 0x7f040000dbc0 con 0x7f040c0faed0 2024-02-20T12:50:32.728 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:32.725+0000 7f040b7fe700 1 -- 172.21.15.64:0/1013837196 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0400011b90 con 0x7f040c0faed0 2024-02-20T12:50:32.728 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:32.725+0000 7f04131e6700 1 -- 172.21.15.64:0/1013837196 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f040c0faed0 msgr2=0x7f040c0fb2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:50:32.729 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:32.725+0000 7f04131e6700 1 --2- 172.21.15.64:0/1013837196 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f040c0faed0 0x7f040c0fb2f0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7f040000b5f0 tx=0x7f0400006ee0).stop 2024-02-20T12:50:32.729 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:32.725+0000 7f04131e6700 1 -- 172.21.15.64:0/1013837196 shutdown_connections 2024-02-20T12:50:32.729 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:32.725+0000 7f04131e6700 1 --2- 172.21.15.64:0/1013837196 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f040c0faed0 0x7f040c0fb2f0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:50:32.729 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:32.725+0000 7f04131e6700 1 -- 172.21.15.64:0/1013837196 >> 172.21.15.64:0/1013837196 conn(0x7f040c0f6680 msgr2=0x7f040c0f8aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:50:32.729 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:32.725+0000 7f04131e6700 1 -- 172.21.15.64:0/1013837196 shutdown_connections 2024-02-20T12:50:32.729 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:32.725+0000 7f04131e6700 1 -- 172.21.15.64:0/1013837196 wait complete. 2024-02-20T12:50:32.729 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:32.726+0000 7f04131e6700 1 Processor -- start 2024-02-20T12:50:32.729 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:32.726+0000 7f04131e6700 1 -- start start 2024-02-20T12:50:32.729 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:32.726+0000 7f04131e6700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f040c0faed0 0x7f040c101150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:50:32.729 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:32.726+0000 7f04131e6700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f040c0ff5d0 con 0x7f040c0faed0 2024-02-20T12:50:32.729 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:32.726+0000 7f0410f82700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f040c0faed0 0x7f040c101150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:50:32.729 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:32.726+0000 7f0410f82700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f040c0faed0 0x7f040c101150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57062/0 (socket says 172.21.15.64:57062) 2024-02-20T12:50:32.729 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:32.726+0000 7f0410f82700 1 -- 172.21.15.64:0/3579796229 learned_addr learned my addr 172.21.15.64:0/3579796229 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-20T12:50:32.729 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:32.727+0000 7f0410f82700 1 -- 172.21.15.64:0/3579796229 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f040000b200 con 0x7f040c0faed0 2024-02-20T12:50:32.731 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:32.727+0000 7f0410f82700 1 --2- 172.21.15.64:0/3579796229 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f040c0faed0 0x7f040c101150 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7f0400000c00 tx=0x7f0400006ee0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:50:32.731 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:32.727+0000 7f04097fa700 1 -- 172.21.15.64:0/3579796229 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f040001b410 con 0x7f040c0faed0 2024-02-20T12:50:32.731 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:32.727+0000 7f04131e6700 1 -- 172.21.15.64:0/3579796229 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f040c0ff770 con 0x7f040c0faed0 2024-02-20T12:50:32.731 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:32.727+0000 7f04097fa700 1 -- 172.21.15.64:0/3579796229 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(25 keys) v1 ==== 997+0+0 (secure 0 0 0) 0x7f040001b9d0 con 0x7f040c0faed0 2024-02-20T12:50:32.731 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:32.727+0000 7f04097fa700 1 -- 172.21.15.64:0/3579796229 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f040001a620 con 0x7f040c0faed0 2024-02-20T12:50:32.731 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:32.727+0000 7f04131e6700 1 -- 172.21.15.64:0/3579796229 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f040c0ffc00 con 0x7f040c0faed0 2024-02-20T12:50:32.731 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:32.728+0000 7f04097fa700 1 -- 172.21.15.64:0/3579796229 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 8) v1 ==== 37095+0+0 (secure 0 0 0) 0x7f040002b070 con 0x7f040c0faed0 2024-02-20T12:50:32.731 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:32.728+0000 7f04097fa700 1 --2- 172.21.15.64:0/3579796229 >> [v2:172.21.15.64:6800/1362908731,v1:172.21.15.64:6801/1362908731] conn(0x7f03f0030ff0 0x7f03f00334b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:50:32.731 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:32.728+0000 7f04131e6700 1 -- 172.21.15.64:0/3579796229 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f040c045bb0 con 0x7f040c0faed0 2024-02-20T12:50:32.731 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:32.728+0000 7f04097fa700 1 -- 172.21.15.64:0/3579796229 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 893+0+0 (secure 0 0 0) 0x7f0400042b50 con 0x7f040c0faed0 2024-02-20T12:50:32.731 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:32.728+0000 7f040bfff700 1 --2- 172.21.15.64:0/3579796229 >> [v2:172.21.15.64:6800/1362908731,v1:172.21.15.64:6801/1362908731] conn(0x7f03f0030ff0 0x7f03f00334b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:50:32.731 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:32.728+0000 7f040bfff700 1 --2- 172.21.15.64:0/3579796229 >> [v2:172.21.15.64:6800/1362908731,v1:172.21.15.64:6801/1362908731] conn(0x7f03f0030ff0 0x7f03f00334b0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 rx=0x7f03f80096a0 tx=0x7f03f8006b40).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:50:32.733 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:32.732+0000 7f04097fa700 1 -- 172.21.15.64:0/3579796229 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+153237 (secure 0 0 0) 0x7f040001b570 con 0x7f040c0faed0 2024-02-20T12:50:32.812 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:32 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:50:31.424220+0000 mon.smithi064 (mon.0) 55 : audit [INF] from='mgr.14116 172.21.15.64:0/330181513' entity='mgr.smithi064.vriyor' 2024-02-20T12:50:32.812 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:32 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cephadm 2024-02-20T12:50:31.424491+0000 mgr.smithi064.vriyor (mgr.14116) 9 : cephadm [INF] Added host smithi064 2024-02-20T12:50:32.812 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:32 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:50:31.424739+0000 mon.smithi064 (mon.0) 56 : audit [DBG] from='mgr.14116 172.21.15.64:0/330181513' entity='mgr.smithi064.vriyor' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-20T12:50:32.902 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:32.901+0000 7f04131e6700 1 -- 172.21.15.64:0/3579796229 --> [v2:172.21.15.64:6800/1362908731,v1:172.21.15.64:6801/1362908731] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "target": ["mon-mgr", ""]}) v1 -- 0x7f040c0ffe40 con 0x7f03f0030ff0 2024-02-20T12:50:32.906 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:32.905+0000 7f04097fa700 1 -- 172.21.15.64:0/3579796229 <== mgr.14116 v2:172.21.15.64:6800/1362908731 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f040c0ffe40 con 0x7f03f0030ff0 2024-02-20T12:50:32.906 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: Scheduled mon update... 2024-02-20T12:50:32.909 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:32.907+0000 7f04131e6700 1 -- 172.21.15.64:0/3579796229 >> [v2:172.21.15.64:6800/1362908731,v1:172.21.15.64:6801/1362908731] conn(0x7f03f0030ff0 msgr2=0x7f03f00334b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:50:32.909 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:32.907+0000 7f04131e6700 1 --2- 172.21.15.64:0/3579796229 >> [v2:172.21.15.64:6800/1362908731,v1:172.21.15.64:6801/1362908731] conn(0x7f03f0030ff0 0x7f03f00334b0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 rx=0x7f03f80096a0 tx=0x7f03f8006b40).stop 2024-02-20T12:50:32.909 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:32.907+0000 7f04131e6700 1 -- 172.21.15.64:0/3579796229 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f040c0faed0 msgr2=0x7f040c101150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:50:32.909 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:32.907+0000 7f04131e6700 1 --2- 172.21.15.64:0/3579796229 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f040c0faed0 0x7f040c101150 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7f0400000c00 tx=0x7f0400006ee0).stop 2024-02-20T12:50:32.909 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:32.907+0000 7f04131e6700 1 -- 172.21.15.64:0/3579796229 shutdown_connections 2024-02-20T12:50:32.909 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:32.907+0000 7f04131e6700 1 --2- 172.21.15.64:0/3579796229 >> [v2:172.21.15.64:6800/1362908731,v1:172.21.15.64:6801/1362908731] conn(0x7f03f0030ff0 0x7f03f00334b0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:50:32.909 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:32.907+0000 7f04131e6700 1 --2- 172.21.15.64:0/3579796229 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f040c0faed0 0x7f040c101150 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:50:32.909 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:32.907+0000 7f04131e6700 1 -- 172.21.15.64:0/3579796229 >> 172.21.15.64:0/3579796229 conn(0x7f040c0f6680 msgr2=0x7f040c0271b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:50:32.910 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:32.907+0000 7f04131e6700 1 -- 172.21.15.64:0/3579796229 shutdown_connections 2024-02-20T12:50:32.910 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:32.907+0000 7f04131e6700 1 -- 172.21.15.64:0/3579796229 wait complete. 2024-02-20T12:50:33.591 INFO:teuthology.orchestra.run.smithi064.stderr:Deploying mgr service with default placement... 2024-02-20T12:50:34.254 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:34.223+0000 7f2739bfe700 1 Processor -- start 2024-02-20T12:50:34.255 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:34.223+0000 7f2739bfe700 1 -- start start 2024-02-20T12:50:34.255 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:34.223+0000 7f2739bfe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f27340fd6a0 0x7f27340fdac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-02-20T12:50:34.255 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:34.223+0000 7f2739bfe700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27340fdfb0 con 0x7f27340fd6a0 2024-02-20T12:50:34.255 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:34.223+0000 7f27337fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f27340fd6a0 0x7f27340fdac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:50:34.255 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:34.223+0000 7f27337fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f27340fd6a0 0x7f27340fdac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57072/0 (socket says 172.21.15.64:57072) 2024-02-20T12:50:34.255 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:34.223+0000 7f27337fe700 1 -- 172.21.15.64:0/1650423464 learned_addr learned my addr 172.21.15.64:0/1650423464 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-20T12:50:34.255 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:34.224+0000 7f27337fe700 1 -- 172.21.15.64:0/1650423464 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27340fe0f0 con 0x7f27340fd6a0 2024-02-20T12:50:34.255 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:34.224+0000 7f27337fe700 1 --2- 172.21.15.64:0/1650423464 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f27340fd6a0 0x7f27340fdac0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7f272800b5f0 tx=0x7f2728006ee0).ready entity=mon.0 client_cookie=f180130358c4135e server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:50:34.255 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:34.224+0000 7f27327fc700 1 -- 172.21.15.64:0/1650423464 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f272800d600 con 0x7f27340fd6a0 2024-02-20T12:50:34.255 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:34.224+0000 7f27327fc700 1 -- 172.21.15.64:0/1650423464 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(25 keys) v1 ==== 997+0+0 (secure 0 0 0) 0x7f272800dbc0 con 0x7f27340fd6a0 2024-02-20T12:50:34.255 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:34.224+0000 7f27327fc700 1 -- 172.21.15.64:0/1650423464 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2728011b90 con 0x7f27340fd6a0 2024-02-20T12:50:34.255 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:34.224+0000 7f2739bfe700 1 -- 172.21.15.64:0/1650423464 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f27340fd6a0 msgr2=0x7f27340fdac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:50:34.255 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:34.224+0000 7f2739bfe700 1 --2- 172.21.15.64:0/1650423464 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f27340fd6a0 0x7f27340fdac0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7f272800b5f0 tx=0x7f2728006ee0).stop 2024-02-20T12:50:34.255 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:34.224+0000 7f2739bfe700 1 -- 172.21.15.64:0/1650423464 shutdown_connections 2024-02-20T12:50:34.255 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:34.224+0000 7f2739bfe700 1 --2- 172.21.15.64:0/1650423464 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f27340fd6a0 0x7f27340fdac0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:50:34.256 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:34.224+0000 7f2739bfe700 1 -- 172.21.15.64:0/1650423464 >> 172.21.15.64:0/1650423464 conn(0x7f27340f5e20 msgr2=0x7f27340f8240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:50:34.256 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:34.224+0000 7f2739bfe700 1 -- 172.21.15.64:0/1650423464 shutdown_connections 2024-02-20T12:50:34.256 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:34.225+0000 7f2739bfe700 1 -- 172.21.15.64:0/1650423464 wait complete. 2024-02-20T12:50:34.256 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:34.225+0000 7f2739bfe700 1 Processor -- start 2024-02-20T12:50:34.256 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:34.225+0000 7f2739bfe700 1 -- start start 2024-02-20T12:50:34.256 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:34.225+0000 7f2739bfe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f27340fd6a0 0x7f273410ad90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:50:34.256 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:34.225+0000 7f2739bfe700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f273410b280 con 0x7f27340fd6a0 2024-02-20T12:50:34.256 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:34.225+0000 7f27337fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f27340fd6a0 0x7f273410ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:50:34.256 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:34.225+0000 7f27337fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f27340fd6a0 0x7f273410ad90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57080/0 (socket says 172.21.15.64:57080) 2024-02-20T12:50:34.256 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:34.225+0000 7f27337fe700 1 -- 172.21.15.64:0/3641033431 learned_addr learned my addr 172.21.15.64:0/3641033431 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-20T12:50:34.256 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:34.226+0000 7f27337fe700 1 -- 172.21.15.64:0/3641033431 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f272800b200 con 0x7f27340fd6a0 2024-02-20T12:50:34.256 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:34.226+0000 7f27337fe700 1 --2- 172.21.15.64:0/3641033431 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f27340fd6a0 0x7f273410ad90 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7f2728000c00 tx=0x7f2728006ee0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:50:34.256 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:34.226+0000 7f271bfff700 1 -- 172.21.15.64:0/3641033431 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f272801b410 con 0x7f27340fd6a0 2024-02-20T12:50:34.256 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:34.226+0000 7f271bfff700 1 -- 172.21.15.64:0/3641033431 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(25 keys) v1 ==== 997+0+0 (secure 0 0 0) 0x7f272801b9d0 con 0x7f27340fd6a0 2024-02-20T12:50:34.257 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:34.226+0000 7f2739bfe700 1 -- 172.21.15.64:0/3641033431 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f273410b480 con 0x7f27340fd6a0 2024-02-20T12:50:34.257 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:34.226+0000 7f271bfff700 1 -- 172.21.15.64:0/3641033431 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f272801a620 con 0x7f27340fd6a0 2024-02-20T12:50:34.257 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:34.226+0000 7f2739bfe700 1 -- 172.21.15.64:0/3641033431 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f273410c090 con 0x7f27340fd6a0 2024-02-20T12:50:34.257 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:34.227+0000 7f271bfff700 1 -- 172.21.15.64:0/3641033431 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 8) v1 ==== 37095+0+0 (secure 0 0 0) 0x7f272802b070 con 0x7f27340fd6a0 2024-02-20T12:50:34.257 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:34.227+0000 7f271bfff700 1 --2- 172.21.15.64:0/3641033431 >> [v2:172.21.15.64:6800/1362908731,v1:172.21.15.64:6801/1362908731] conn(0x7f2720030fa0 0x7f2720033460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:50:34.257 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:34.227+0000 7f271bfff700 1 -- 172.21.15.64:0/3641033431 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 893+0+0 (secure 0 0 0) 0x7f2728042c70 con 0x7f27340fd6a0 2024-02-20T12:50:34.257 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:34.227+0000 7f2739bfe700 1 -- 172.21.15.64:0/3641033431 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27340591e0 con 0x7f27340fd6a0 2024-02-20T12:50:34.257 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:34.227+0000 7f2732ffd700 1 --2- 172.21.15.64:0/3641033431 >> [v2:172.21.15.64:6800/1362908731,v1:172.21.15.64:6801/1362908731] conn(0x7f2720030fa0 0x7f2720033460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:50:34.257 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:34.227+0000 7f2732ffd700 1 --2- 172.21.15.64:0/3641033431 >> [v2:172.21.15.64:6800/1362908731,v1:172.21.15.64:6801/1362908731] conn(0x7f2720030fa0 0x7f2720033460 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 rx=0x7f27240096a0 tx=0x7f2724006b40).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:50:34.257 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:34.231+0000 7f271bfff700 1 -- 172.21.15.64:0/3641033431 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+153237 (secure 0 0 0) 0x7f272801f020 con 0x7f27340fd6a0 2024-02-20T12:50:34.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:33 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:50:32.903171+0000 mgr.smithi064.vriyor (mgr.14116) 10 : audit [DBG] from='client.14136 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "target": ["mon-mgr", ""]}]: dispatch 2024-02-20T12:50:34.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:33 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cephadm 2024-02-20T12:50:32.903884+0000 mgr.smithi064.vriyor (mgr.14116) 11 : cephadm [INF] Saving service mon spec with placement count:5 2024-02-20T12:50:34.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:33 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:50:32.906164+0000 mon.smithi064 (mon.0) 57 : audit [INF] from='mgr.14116 172.21.15.64:0/330181513' entity='mgr.smithi064.vriyor' 2024-02-20T12:50:34.403 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:34.401+0000 7f2739bfe700 1 -- 172.21.15.64:0/3641033431 --> [v2:172.21.15.64:6800/1362908731,v1:172.21.15.64:6801/1362908731] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "target": ["mon-mgr", ""]}) v1 -- 0x7f27340fab80 con 0x7f2720030fa0 2024-02-20T12:50:34.406 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:34.405+0000 7f271bfff700 1 -- 172.21.15.64:0/3641033431 <== mgr.14116 v2:172.21.15.64:6800/1362908731 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f27340fab80 con 0x7f2720030fa0 2024-02-20T12:50:34.406 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: Scheduled mgr update... 2024-02-20T12:50:34.409 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:34.407+0000 7f2739bfe700 1 -- 172.21.15.64:0/3641033431 >> [v2:172.21.15.64:6800/1362908731,v1:172.21.15.64:6801/1362908731] conn(0x7f2720030fa0 msgr2=0x7f2720033460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:50:34.409 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:34.407+0000 7f2739bfe700 1 --2- 172.21.15.64:0/3641033431 >> [v2:172.21.15.64:6800/1362908731,v1:172.21.15.64:6801/1362908731] conn(0x7f2720030fa0 0x7f2720033460 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 rx=0x7f27240096a0 tx=0x7f2724006b40).stop 2024-02-20T12:50:34.409 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:34.407+0000 7f2739bfe700 1 -- 172.21.15.64:0/3641033431 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f27340fd6a0 msgr2=0x7f273410ad90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:50:34.409 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:34.407+0000 7f2739bfe700 1 --2- 172.21.15.64:0/3641033431 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f27340fd6a0 0x7f273410ad90 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7f2728000c00 tx=0x7f2728006ee0).stop 2024-02-20T12:50:34.409 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:34.407+0000 7f2739bfe700 1 -- 172.21.15.64:0/3641033431 shutdown_connections 2024-02-20T12:50:34.409 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:34.407+0000 7f2739bfe700 1 --2- 172.21.15.64:0/3641033431 >> [v2:172.21.15.64:6800/1362908731,v1:172.21.15.64:6801/1362908731] conn(0x7f2720030fa0 0x7f2720033460 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:50:34.409 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:34.407+0000 7f2739bfe700 1 --2- 172.21.15.64:0/3641033431 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f27340fd6a0 0x7f273410ad90 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:50:34.409 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:34.407+0000 7f2739bfe700 1 -- 172.21.15.64:0/3641033431 >> 172.21.15.64:0/3641033431 conn(0x7f27340f5e20 msgr2=0x7f27340ff5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:50:34.409 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:34.408+0000 7f2739bfe700 1 -- 172.21.15.64:0/3641033431 shutdown_connections 2024-02-20T12:50:34.409 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:34.408+0000 7f2739bfe700 1 -- 172.21.15.64:0/3641033431 wait complete. 2024-02-20T12:50:35.065 INFO:teuthology.orchestra.run.smithi064.stderr:Deploying crash service with default placement... 2024-02-20T12:50:35.642 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:35.613+0000 7f8c86e3c700 1 Processor -- start 2024-02-20T12:50:35.642 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:35.613+0000 7f8c86e3c700 1 -- start start 2024-02-20T12:50:35.642 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:35.614+0000 7f8c86e3c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8c800fbfc0 0x7f8c800fc3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-02-20T12:50:35.642 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:35.614+0000 7f8c86e3c700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c800fc8d0 con 0x7f8c800fbfc0 2024-02-20T12:50:35.642 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:35.614+0000 7f8c84bd8700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8c800fbfc0 0x7f8c800fc3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:50:35.642 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:35.614+0000 7f8c84bd8700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8c800fbfc0 0x7f8c800fc3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57086/0 (socket says 172.21.15.64:57086) 2024-02-20T12:50:35.642 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:35.614+0000 7f8c84bd8700 1 -- 172.21.15.64:0/1691272138 learned_addr learned my addr 172.21.15.64:0/1691272138 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-20T12:50:35.642 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:35.614+0000 7f8c84bd8700 1 -- 172.21.15.64:0/1691272138 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c800fca10 con 0x7f8c800fbfc0 2024-02-20T12:50:35.642 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:35.614+0000 7f8c84bd8700 1 --2- 172.21.15.64:0/1691272138 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8c800fbfc0 0x7f8c800fc3e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7f8c7400b5f0 tx=0x7f8c74006ee0).ready entity=mon.0 client_cookie=2fb8e3c3e7ffd6e8 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:50:35.642 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:35.614+0000 7f8c7f7fe700 1 -- 172.21.15.64:0/1691272138 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8c7400d600 con 0x7f8c800fbfc0 2024-02-20T12:50:35.643 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:35.614+0000 7f8c7f7fe700 1 -- 172.21.15.64:0/1691272138 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(25 keys) v1 ==== 997+0+0 (secure 0 0 0) 0x7f8c7400dbc0 con 0x7f8c800fbfc0 2024-02-20T12:50:35.643 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:35.614+0000 7f8c7f7fe700 1 -- 172.21.15.64:0/1691272138 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8c74011b90 con 0x7f8c800fbfc0 2024-02-20T12:50:35.643 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:35.615+0000 7f8c86e3c700 1 -- 172.21.15.64:0/1691272138 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8c800fbfc0 msgr2=0x7f8c800fc3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:50:35.643 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:35.615+0000 7f8c86e3c700 1 --2- 172.21.15.64:0/1691272138 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8c800fbfc0 0x7f8c800fc3e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7f8c7400b5f0 tx=0x7f8c74006ee0).stop 2024-02-20T12:50:35.643 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:35.615+0000 7f8c86e3c700 1 -- 172.21.15.64:0/1691272138 shutdown_connections 2024-02-20T12:50:35.643 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:35.615+0000 7f8c86e3c700 1 --2- 172.21.15.64:0/1691272138 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8c800fbfc0 0x7f8c800fc3e0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:50:35.643 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:35.615+0000 7f8c86e3c700 1 -- 172.21.15.64:0/1691272138 >> 172.21.15.64:0/1691272138 conn(0x7f8c800f7770 msgr2=0x7f8c800f9b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:50:35.643 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:35.615+0000 7f8c86e3c700 1 -- 172.21.15.64:0/1691272138 shutdown_connections 2024-02-20T12:50:35.643 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:35.615+0000 7f8c86e3c700 1 -- 172.21.15.64:0/1691272138 wait complete. 2024-02-20T12:50:35.643 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:35.616+0000 7f8c86e3c700 1 Processor -- start 2024-02-20T12:50:35.643 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:35.616+0000 7f8c86e3c700 1 -- start start 2024-02-20T12:50:35.643 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:35.616+0000 7f8c86e3c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8c800fbfc0 0x7f8c8010cea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:50:35.643 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:35.616+0000 7f8c86e3c700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c8010d390 con 0x7f8c800fbfc0 2024-02-20T12:50:35.643 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:35.616+0000 7f8c84bd8700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8c800fbfc0 0x7f8c8010cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:50:35.643 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:35.616+0000 7f8c84bd8700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8c800fbfc0 0x7f8c8010cea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57100/0 (socket says 172.21.15.64:57100) 2024-02-20T12:50:35.644 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:35.616+0000 7f8c84bd8700 1 -- 172.21.15.64:0/519180062 learned_addr learned my addr 172.21.15.64:0/519180062 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-20T12:50:35.644 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:35.616+0000 7f8c84bd8700 1 -- 172.21.15.64:0/519180062 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c7400b200 con 0x7f8c800fbfc0 2024-02-20T12:50:35.644 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:35.616+0000 7f8c84bd8700 1 --2- 172.21.15.64:0/519180062 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8c800fbfc0 0x7f8c8010cea0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7f8c74006800 tx=0x7f8c74006ee0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:50:35.644 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:35.617+0000 7f8c7d7fa700 1 -- 172.21.15.64:0/519180062 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8c7401b410 con 0x7f8c800fbfc0 2024-02-20T12:50:35.644 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:35.617+0000 7f8c7d7fa700 1 -- 172.21.15.64:0/519180062 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(25 keys) v1 ==== 997+0+0 (secure 0 0 0) 0x7f8c7401b9d0 con 0x7f8c800fbfc0 2024-02-20T12:50:35.644 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:35.617+0000 7f8c86e3c700 1 -- 172.21.15.64:0/519180062 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c8010d590 con 0x7f8c800fbfc0 2024-02-20T12:50:35.644 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:35.617+0000 7f8c7d7fa700 1 -- 172.21.15.64:0/519180062 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8c7401a840 con 0x7f8c800fbfc0 2024-02-20T12:50:35.644 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:35.617+0000 7f8c86e3c700 1 -- 172.21.15.64:0/519180062 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c8010da20 con 0x7f8c800fbfc0 2024-02-20T12:50:35.644 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:35.617+0000 7f8c7d7fa700 1 -- 172.21.15.64:0/519180062 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 8) v1 ==== 37095+0+0 (secure 0 0 0) 0x7f8c7400f070 con 0x7f8c800fbfc0 2024-02-20T12:50:35.644 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:35.618+0000 7f8c7d7fa700 1 --2- 172.21.15.64:0/519180062 >> [v2:172.21.15.64:6800/1362908731,v1:172.21.15.64:6801/1362908731] conn(0x7f8c64030ff0 0x7f8c640334b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:50:35.644 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:35.618+0000 7f8c7d7fa700 1 -- 172.21.15.64:0/519180062 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 893+0+0 (secure 0 0 0) 0x7f8c74043a10 con 0x7f8c800fbfc0 2024-02-20T12:50:35.644 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:35.618+0000 7f8c7ffff700 1 --2- 172.21.15.64:0/519180062 >> [v2:172.21.15.64:6800/1362908731,v1:172.21.15.64:6801/1362908731] conn(0x7f8c64030ff0 0x7f8c640334b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:50:35.644 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:35.618+0000 7f8c86e3c700 1 -- 172.21.15.64:0/519180062 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c80045bb0 con 0x7f8c800fbfc0 2024-02-20T12:50:35.644 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:35.618+0000 7f8c7ffff700 1 --2- 172.21.15.64:0/519180062 >> [v2:172.21.15.64:6800/1362908731,v1:172.21.15.64:6801/1362908731] conn(0x7f8c64030ff0 0x7f8c640334b0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 rx=0x7f8c6c0096a0 tx=0x7f8c6c006b40).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:50:35.645 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:35.622+0000 7f8c7d7fa700 1 -- 172.21.15.64:0/519180062 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+153237 (secure 0 0 0) 0x7f8c74016070 con 0x7f8c800fbfc0 2024-02-20T12:50:35.794 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:35.793+0000 7f8c86e3c700 1 -- 172.21.15.64:0/519180062 --> [v2:172.21.15.64:6800/1362908731,v1:172.21.15.64:6801/1362908731] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "crash", "target": ["mon-mgr", ""]}) v1 -- 0x7f8c801070d0 con 0x7f8c64030ff0 2024-02-20T12:50:35.798 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:35.797+0000 7f8c7d7fa700 1 -- 172.21.15.64:0/519180062 <== mgr.14116 v2:172.21.15.64:6800/1362908731 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+26 (secure 0 0 0) 0x7f8c801070d0 con 0x7f8c64030ff0 2024-02-20T12:50:35.798 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: Scheduled crash update... 2024-02-20T12:50:35.800 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:35.799+0000 7f8c86e3c700 1 -- 172.21.15.64:0/519180062 >> [v2:172.21.15.64:6800/1362908731,v1:172.21.15.64:6801/1362908731] conn(0x7f8c64030ff0 msgr2=0x7f8c640334b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:50:35.800 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:35.799+0000 7f8c86e3c700 1 --2- 172.21.15.64:0/519180062 >> [v2:172.21.15.64:6800/1362908731,v1:172.21.15.64:6801/1362908731] conn(0x7f8c64030ff0 0x7f8c640334b0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 rx=0x7f8c6c0096a0 tx=0x7f8c6c006b40).stop 2024-02-20T12:50:35.801 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:35.799+0000 7f8c86e3c700 1 -- 172.21.15.64:0/519180062 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8c800fbfc0 msgr2=0x7f8c8010cea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:50:35.801 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:35.799+0000 7f8c86e3c700 1 --2- 172.21.15.64:0/519180062 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8c800fbfc0 0x7f8c8010cea0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7f8c74006800 tx=0x7f8c74006ee0).stop 2024-02-20T12:50:35.801 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:35.799+0000 7f8c86e3c700 1 -- 172.21.15.64:0/519180062 shutdown_connections 2024-02-20T12:50:35.801 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:35.799+0000 7f8c86e3c700 1 --2- 172.21.15.64:0/519180062 >> [v2:172.21.15.64:6800/1362908731,v1:172.21.15.64:6801/1362908731] conn(0x7f8c64030ff0 0x7f8c640334b0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:50:35.801 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:35.799+0000 7f8c86e3c700 1 --2- 172.21.15.64:0/519180062 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8c800fbfc0 0x7f8c8010cea0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:50:35.801 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:35.799+0000 7f8c86e3c700 1 -- 172.21.15.64:0/519180062 >> 172.21.15.64:0/519180062 conn(0x7f8c800f7770 msgr2=0x7f8c800f9140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:50:35.801 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:35.799+0000 7f8c86e3c700 1 -- 172.21.15.64:0/519180062 shutdown_connections 2024-02-20T12:50:35.801 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:35.799+0000 7f8c86e3c700 1 -- 172.21.15.64:0/519180062 wait complete. 2024-02-20T12:50:35.812 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:35 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:50:34.403604+0000 mgr.smithi064.vriyor (mgr.14116) 12 : audit [DBG] from='client.14138 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "target": ["mon-mgr", ""]}]: dispatch 2024-02-20T12:50:35.812 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:35 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cephadm 2024-02-20T12:50:34.404112+0000 mgr.smithi064.vriyor (mgr.14116) 13 : cephadm [INF] Saving service mgr spec with placement count:2 2024-02-20T12:50:35.812 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:35 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:50:34.406206+0000 mon.smithi064 (mon.0) 58 : audit [INF] from='mgr.14116 172.21.15.64:0/330181513' entity='mgr.smithi064.vriyor' 2024-02-20T12:50:36.405 INFO:teuthology.orchestra.run.smithi064.stderr:Deploying prometheus service with default placement... 2024-02-20T12:50:37.000 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:36.989+0000 7f0ddbaf5700 1 Processor -- start 2024-02-20T12:50:37.000 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:36.989+0000 7f0ddbaf5700 1 -- start start 2024-02-20T12:50:37.001 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:36.989+0000 7f0ddbaf5700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0dd4071c60 0x7f0dd4070160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-02-20T12:50:37.001 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:36.989+0000 7f0ddbaf5700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dd4072080 con 0x7f0dd4071c60 2024-02-20T12:50:37.001 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:36.989+0000 7f0dd9891700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0dd4071c60 0x7f0dd4070160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:50:37.001 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:36.990+0000 7f0dd9891700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0dd4071c60 0x7f0dd4070160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57102/0 (socket says 172.21.15.64:57102) 2024-02-20T12:50:37.001 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:36.990+0000 7f0dd9891700 1 -- 172.21.15.64:0/2500358890 learned_addr learned my addr 172.21.15.64:0/2500358890 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-20T12:50:37.001 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:36.990+0000 7f0dd9891700 1 -- 172.21.15.64:0/2500358890 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0dd4070650 con 0x7f0dd4071c60 2024-02-20T12:50:37.001 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:36.990+0000 7f0dd9891700 1 --2- 172.21.15.64:0/2500358890 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0dd4071c60 0x7f0dd4070160 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7f0dc400b5f0 tx=0x7f0dc4006ee0).ready entity=mon.0 client_cookie=50a682e2051cd4a7 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:50:37.001 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:36.990+0000 7f0dd888f700 1 -- 172.21.15.64:0/2500358890 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0dc400d600 con 0x7f0dd4071c60 2024-02-20T12:50:37.001 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:36.990+0000 7f0dd888f700 1 -- 172.21.15.64:0/2500358890 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(25 keys) v1 ==== 997+0+0 (secure 0 0 0) 0x7f0dc400dbc0 con 0x7f0dd4071c60 2024-02-20T12:50:37.001 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:36.990+0000 7f0dd888f700 1 -- 172.21.15.64:0/2500358890 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0dc4011b90 con 0x7f0dd4071c60 2024-02-20T12:50:37.001 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:36.991+0000 7f0ddbaf5700 1 -- 172.21.15.64:0/2500358890 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0dd4071c60 msgr2=0x7f0dd4070160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:50:37.001 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:36.991+0000 7f0ddbaf5700 1 --2- 172.21.15.64:0/2500358890 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0dd4071c60 0x7f0dd4070160 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7f0dc400b5f0 tx=0x7f0dc4006ee0).stop 2024-02-20T12:50:37.001 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:36.991+0000 7f0ddbaf5700 1 -- 172.21.15.64:0/2500358890 shutdown_connections 2024-02-20T12:50:37.001 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:36.991+0000 7f0ddbaf5700 1 --2- 172.21.15.64:0/2500358890 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0dd4071c60 0x7f0dd4070160 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:50:37.001 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:36.991+0000 7f0ddbaf5700 1 -- 172.21.15.64:0/2500358890 >> 172.21.15.64:0/2500358890 conn(0x7f0dd40f7770 msgr2=0x7f0dd40f9b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:50:37.002 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:36.991+0000 7f0ddbaf5700 1 -- 172.21.15.64:0/2500358890 shutdown_connections 2024-02-20T12:50:37.002 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:36.991+0000 7f0ddbaf5700 1 -- 172.21.15.64:0/2500358890 wait complete. 2024-02-20T12:50:37.002 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:36.991+0000 7f0ddbaf5700 1 Processor -- start 2024-02-20T12:50:37.002 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:36.991+0000 7f0ddbaf5700 1 -- start start 2024-02-20T12:50:37.002 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:36.991+0000 7f0ddbaf5700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0dd4071c60 0x7f0dd410ced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:50:37.002 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:36.991+0000 7f0ddbaf5700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dd410d3c0 con 0x7f0dd4071c60 2024-02-20T12:50:37.002 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:36.991+0000 7f0dd9891700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0dd4071c60 0x7f0dd410ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:50:37.002 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:36.992+0000 7f0dd9891700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0dd4071c60 0x7f0dd410ced0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57118/0 (socket says 172.21.15.64:57118) 2024-02-20T12:50:37.002 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:36.992+0000 7f0dd9891700 1 -- 172.21.15.64:0/366049326 learned_addr learned my addr 172.21.15.64:0/366049326 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-20T12:50:37.002 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:36.992+0000 7f0dd9891700 1 -- 172.21.15.64:0/366049326 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0dc400b200 con 0x7f0dd4071c60 2024-02-20T12:50:37.002 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:36.992+0000 7f0dd9891700 1 --2- 172.21.15.64:0/366049326 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0dd4071c60 0x7f0dd410ced0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7f0dc400e040 tx=0x7f0dc4006ee0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:50:37.002 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:36.992+0000 7f0dc27fc700 1 -- 172.21.15.64:0/366049326 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0dc401b570 con 0x7f0dd4071c60 2024-02-20T12:50:37.002 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:36.992+0000 7f0dc27fc700 1 -- 172.21.15.64:0/366049326 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(25 keys) v1 ==== 997+0+0 (secure 0 0 0) 0x7f0dc401bb30 con 0x7f0dd4071c60 2024-02-20T12:50:37.003 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:36.992+0000 7f0ddbaf5700 1 -- 172.21.15.64:0/366049326 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0dd410d5c0 con 0x7f0dd4071c60 2024-02-20T12:50:37.003 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:36.992+0000 7f0dc27fc700 1 -- 172.21.15.64:0/366049326 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0dc401a820 con 0x7f0dd4071c60 2024-02-20T12:50:37.003 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:36.992+0000 7f0ddbaf5700 1 -- 172.21.15.64:0/366049326 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0dd410da50 con 0x7f0dd4071c60 2024-02-20T12:50:37.003 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:36.993+0000 7f0dc27fc700 1 -- 172.21.15.64:0/366049326 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 8) v1 ==== 37095+0+0 (secure 0 0 0) 0x7f0dc402b070 con 0x7f0dd4071c60 2024-02-20T12:50:37.003 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:36.993+0000 7f0ddbaf5700 1 -- 172.21.15.64:0/366049326 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0dd4045bb0 con 0x7f0dd4071c60 2024-02-20T12:50:37.003 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:36.993+0000 7f0dc27fc700 1 --2- 172.21.15.64:0/366049326 >> [v2:172.21.15.64:6800/1362908731,v1:172.21.15.64:6801/1362908731] conn(0x7f0dbc030ff0 0x7f0dbc0334b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:50:37.003 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:36.993+0000 7f0dc27fc700 1 -- 172.21.15.64:0/366049326 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 893+0+0 (secure 0 0 0) 0x7f0dc4042db0 con 0x7f0dd4071c60 2024-02-20T12:50:37.003 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:36.993+0000 7f0dd9090700 1 --2- 172.21.15.64:0/366049326 >> [v2:172.21.15.64:6800/1362908731,v1:172.21.15.64:6801/1362908731] conn(0x7f0dbc030ff0 0x7f0dbc0334b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:50:37.003 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:36.993+0000 7f0dd9090700 1 --2- 172.21.15.64:0/366049326 >> [v2:172.21.15.64:6800/1362908731,v1:172.21.15.64:6801/1362908731] conn(0x7f0dbc030ff0 0x7f0dbc0334b0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 rx=0x7f0dd00096a0 tx=0x7f0dd0006b40).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:50:37.003 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:36.997+0000 7f0dc27fc700 1 -- 172.21.15.64:0/366049326 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+153237 (secure 0 0 0) 0x7f0dc402b6c0 con 0x7f0dd4071c60 2024-02-20T12:50:37.062 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:36 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:50:35.795122+0000 mgr.smithi064.vriyor (mgr.14116) 14 : audit [DBG] from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "crash", "target": ["mon-mgr", ""]}]: dispatch 2024-02-20T12:50:37.062 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:36 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cephadm 2024-02-20T12:50:35.795733+0000 mgr.smithi064.vriyor (mgr.14116) 15 : cephadm [INF] Saving service crash spec with placement * 2024-02-20T12:50:37.062 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:36 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:50:35.798074+0000 mon.smithi064 (mon.0) 59 : audit [INF] from='mgr.14116 172.21.15.64:0/330181513' entity='mgr.smithi064.vriyor' 2024-02-20T12:50:37.169 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:37.167+0000 7f0ddbaf5700 1 -- 172.21.15.64:0/366049326 --> [v2:172.21.15.64:6800/1362908731,v1:172.21.15.64:6801/1362908731] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "prometheus", "target": ["mon-mgr", ""]}) v1 -- 0x7f0dd410e370 con 0x7f0dbc030ff0 2024-02-20T12:50:37.172 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:37.171+0000 7f0dc27fc700 1 -- 172.21.15.64:0/366049326 <== mgr.14116 v2:172.21.15.64:6800/1362908731 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+31 (secure 0 0 0) 0x7f0dd410e370 con 0x7f0dbc030ff0 2024-02-20T12:50:37.172 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: Scheduled prometheus update... 2024-02-20T12:50:37.175 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:37.173+0000 7f0ddbaf5700 1 -- 172.21.15.64:0/366049326 >> [v2:172.21.15.64:6800/1362908731,v1:172.21.15.64:6801/1362908731] conn(0x7f0dbc030ff0 msgr2=0x7f0dbc0334b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:50:37.175 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:37.173+0000 7f0ddbaf5700 1 --2- 172.21.15.64:0/366049326 >> [v2:172.21.15.64:6800/1362908731,v1:172.21.15.64:6801/1362908731] conn(0x7f0dbc030ff0 0x7f0dbc0334b0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 rx=0x7f0dd00096a0 tx=0x7f0dd0006b40).stop 2024-02-20T12:50:37.175 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:37.173+0000 7f0ddbaf5700 1 -- 172.21.15.64:0/366049326 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0dd4071c60 msgr2=0x7f0dd410ced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:50:37.175 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:37.173+0000 7f0ddbaf5700 1 --2- 172.21.15.64:0/366049326 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0dd4071c60 0x7f0dd410ced0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7f0dc400e040 tx=0x7f0dc4006ee0).stop 2024-02-20T12:50:37.175 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:37.173+0000 7f0ddbaf5700 1 -- 172.21.15.64:0/366049326 shutdown_connections 2024-02-20T12:50:37.175 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:37.173+0000 7f0ddbaf5700 1 --2- 172.21.15.64:0/366049326 >> [v2:172.21.15.64:6800/1362908731,v1:172.21.15.64:6801/1362908731] conn(0x7f0dbc030ff0 0x7f0dbc0334b0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:50:37.175 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:37.173+0000 7f0ddbaf5700 1 --2- 172.21.15.64:0/366049326 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0dd4071c60 0x7f0dd410ced0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:50:37.175 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:37.173+0000 7f0ddbaf5700 1 -- 172.21.15.64:0/366049326 >> 172.21.15.64:0/366049326 conn(0x7f0dd40f7770 msgr2=0x7f0dd40f9170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:50:37.175 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:37.173+0000 7f0ddbaf5700 1 -- 172.21.15.64:0/366049326 shutdown_connections 2024-02-20T12:50:37.176 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:37.173+0000 7f0ddbaf5700 1 -- 172.21.15.64:0/366049326 wait complete. 2024-02-20T12:50:37.781 INFO:teuthology.orchestra.run.smithi064.stderr:Deploying grafana service with default placement... 2024-02-20T12:50:38.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:38 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:50:37.169579+0000 mgr.smithi064.vriyor (mgr.14116) 16 : audit [DBG] from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "target": ["mon-mgr", ""]}]: dispatch 2024-02-20T12:50:38.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:38 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cephadm 2024-02-20T12:50:37.170115+0000 mgr.smithi064.vriyor (mgr.14116) 17 : cephadm [INF] Saving service prometheus spec with placement count:1 2024-02-20T12:50:38.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:38 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:50:37.172386+0000 mon.smithi064 (mon.0) 60 : audit [INF] from='mgr.14116 172.21.15.64:0/330181513' entity='mgr.smithi064.vriyor' 2024-02-20T12:50:38.362 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:38.356+0000 7f3dd8774700 1 Processor -- start 2024-02-20T12:50:38.362 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:38.356+0000 7f3dd8774700 1 -- start start 2024-02-20T12:50:38.362 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:38.356+0000 7f3dd8774700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3dd00faed0 0x7f3dd00fb2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-02-20T12:50:38.362 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:38.356+0000 7f3dd8774700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dd00fb7e0 con 0x7f3dd00faed0 2024-02-20T12:50:38.362 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:38.356+0000 7f3dd6510700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3dd00faed0 0x7f3dd00fb2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:50:38.362 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:38.356+0000 7f3dd6510700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3dd00faed0 0x7f3dd00fb2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57128/0 (socket says 172.21.15.64:57128) 2024-02-20T12:50:38.363 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:38.356+0000 7f3dd6510700 1 -- 172.21.15.64:0/3189420920 learned_addr learned my addr 172.21.15.64:0/3189420920 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-20T12:50:38.363 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:38.357+0000 7f3dd6510700 1 -- 172.21.15.64:0/3189420920 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3dd00fb920 con 0x7f3dd00faed0 2024-02-20T12:50:38.363 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:38.357+0000 7f3dd6510700 1 --2- 172.21.15.64:0/3189420920 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3dd00faed0 0x7f3dd00fb2f0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7f3dcc00b5f0 tx=0x7f3dcc006ee0).ready entity=mon.0 client_cookie=d1f803211b4e3267 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:50:38.363 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:38.357+0000 7f3dd550e700 1 -- 172.21.15.64:0/3189420920 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3dcc00d600 con 0x7f3dd00faed0 2024-02-20T12:50:38.363 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:38.357+0000 7f3dd550e700 1 -- 172.21.15.64:0/3189420920 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(25 keys) v1 ==== 997+0+0 (secure 0 0 0) 0x7f3dcc00dbc0 con 0x7f3dd00faed0 2024-02-20T12:50:38.363 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:38.357+0000 7f3dd550e700 1 -- 172.21.15.64:0/3189420920 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3dcc011b90 con 0x7f3dd00faed0 2024-02-20T12:50:38.363 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:38.358+0000 7f3dd8774700 1 -- 172.21.15.64:0/3189420920 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3dd00faed0 msgr2=0x7f3dd00fb2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:50:38.363 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:38.358+0000 7f3dd8774700 1 --2- 172.21.15.64:0/3189420920 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3dd00faed0 0x7f3dd00fb2f0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7f3dcc00b5f0 tx=0x7f3dcc006ee0).stop 2024-02-20T12:50:38.363 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:38.358+0000 7f3dd8774700 1 -- 172.21.15.64:0/3189420920 shutdown_connections 2024-02-20T12:50:38.363 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:38.358+0000 7f3dd8774700 1 --2- 172.21.15.64:0/3189420920 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3dd00faed0 0x7f3dd00fb2f0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:50:38.363 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:38.358+0000 7f3dd8774700 1 -- 172.21.15.64:0/3189420920 >> 172.21.15.64:0/3189420920 conn(0x7f3dd00f6680 msgr2=0x7f3dd00f8aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:50:38.363 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:38.358+0000 7f3dd8774700 1 -- 172.21.15.64:0/3189420920 shutdown_connections 2024-02-20T12:50:38.363 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:38.358+0000 7f3dd8774700 1 -- 172.21.15.64:0/3189420920 wait complete. 2024-02-20T12:50:38.364 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:38.358+0000 7f3dd8774700 1 Processor -- start 2024-02-20T12:50:38.364 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:38.359+0000 7f3dd8774700 1 -- start start 2024-02-20T12:50:38.364 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:38.359+0000 7f3dd8774700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3dd00faed0 0x7f3dd0108af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:50:38.364 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:38.359+0000 7f3dd8774700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dd0108fe0 con 0x7f3dd00faed0 2024-02-20T12:50:38.364 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:38.359+0000 7f3dd6510700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3dd00faed0 0x7f3dd0108af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:50:38.364 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:38.359+0000 7f3dd6510700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3dd00faed0 0x7f3dd0108af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57130/0 (socket says 172.21.15.64:57130) 2024-02-20T12:50:38.364 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:38.359+0000 7f3dd6510700 1 -- 172.21.15.64:0/3447491507 learned_addr learned my addr 172.21.15.64:0/3447491507 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-20T12:50:38.364 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:38.359+0000 7f3dd6510700 1 -- 172.21.15.64:0/3447491507 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3dcc00b200 con 0x7f3dd00faed0 2024-02-20T12:50:38.364 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:38.359+0000 7f3dd6510700 1 --2- 172.21.15.64:0/3447491507 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3dd00faed0 0x7f3dd0108af0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7f3dcc006800 tx=0x7f3dcc006ee0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:50:38.364 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:38.359+0000 7f3dbeffd700 1 -- 172.21.15.64:0/3447491507 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3dcc00de50 con 0x7f3dd00faed0 2024-02-20T12:50:38.364 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:38.359+0000 7f3dbeffd700 1 -- 172.21.15.64:0/3447491507 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(25 keys) v1 ==== 997+0+0 (secure 0 0 0) 0x7f3dcc01b870 con 0x7f3dd00faed0 2024-02-20T12:50:38.364 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:38.359+0000 7f3dd8774700 1 -- 172.21.15.64:0/3447491507 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3dd01091e0 con 0x7f3dd00faed0 2024-02-20T12:50:38.364 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:38.360+0000 7f3dbeffd700 1 -- 172.21.15.64:0/3447491507 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3dcc01a7a0 con 0x7f3dd00faed0 2024-02-20T12:50:38.364 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:38.360+0000 7f3dd8774700 1 -- 172.21.15.64:0/3447491507 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3dd0109670 con 0x7f3dd00faed0 2024-02-20T12:50:38.364 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:38.360+0000 7f3dbeffd700 1 -- 172.21.15.64:0/3447491507 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 8) v1 ==== 37095+0+0 (secure 0 0 0) 0x7f3dcc00f070 con 0x7f3dd00faed0 2024-02-20T12:50:38.365 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:38.360+0000 7f3dbeffd700 1 --2- 172.21.15.64:0/3447491507 >> [v2:172.21.15.64:6800/1362908731,v1:172.21.15.64:6801/1362908731] conn(0x7f3db8033240 0x7f3db8035700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:50:38.365 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:38.361+0000 7f3dbeffd700 1 -- 172.21.15.64:0/3447491507 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 893+0+0 (secure 0 0 0) 0x7f3dcc042b40 con 0x7f3dd00faed0 2024-02-20T12:50:38.365 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:38.361+0000 7f3dd8774700 1 -- 172.21.15.64:0/3447491507 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3dd0045bb0 con 0x7f3dd00faed0 2024-02-20T12:50:38.365 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:38.361+0000 7f3dd5d0f700 1 --2- 172.21.15.64:0/3447491507 >> [v2:172.21.15.64:6800/1362908731,v1:172.21.15.64:6801/1362908731] conn(0x7f3db8033240 0x7f3db8035700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:50:38.365 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:38.361+0000 7f3dd5d0f700 1 --2- 172.21.15.64:0/3447491507 >> [v2:172.21.15.64:6800/1362908731,v1:172.21.15.64:6801/1362908731] conn(0x7f3db8033240 0x7f3db8035700 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 rx=0x7f3dc00096a0 tx=0x7f3dc0006b40).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:50:38.366 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:38.365+0000 7f3dbeffd700 1 -- 172.21.15.64:0/3447491507 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+153237 (secure 0 0 0) 0x7f3dcc016030 con 0x7f3dd00faed0 2024-02-20T12:50:38.534 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:38.533+0000 7f3dd8774700 1 -- 172.21.15.64:0/3447491507 --> [v2:172.21.15.64:6800/1362908731,v1:172.21.15.64:6801/1362908731] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "grafana", "target": ["mon-mgr", ""]}) v1 -- 0x7f3dd0109f90 con 0x7f3db8033240 2024-02-20T12:50:38.539 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:38.537+0000 7f3dbeffd700 1 -- 172.21.15.64:0/3447491507 <== mgr.14116 v2:172.21.15.64:6800/1362908731 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+28 (secure 0 0 0) 0x7f3dd0109f90 con 0x7f3db8033240 2024-02-20T12:50:38.539 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: Scheduled grafana update... 2024-02-20T12:50:38.541 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:38.539+0000 7f3dd8774700 1 -- 172.21.15.64:0/3447491507 >> [v2:172.21.15.64:6800/1362908731,v1:172.21.15.64:6801/1362908731] conn(0x7f3db8033240 msgr2=0x7f3db8035700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:50:38.541 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:38.539+0000 7f3dd8774700 1 --2- 172.21.15.64:0/3447491507 >> [v2:172.21.15.64:6800/1362908731,v1:172.21.15.64:6801/1362908731] conn(0x7f3db8033240 0x7f3db8035700 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 rx=0x7f3dc00096a0 tx=0x7f3dc0006b40).stop 2024-02-20T12:50:38.541 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:38.539+0000 7f3dd8774700 1 -- 172.21.15.64:0/3447491507 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3dd00faed0 msgr2=0x7f3dd0108af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:50:38.541 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:38.539+0000 7f3dd8774700 1 --2- 172.21.15.64:0/3447491507 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3dd00faed0 0x7f3dd0108af0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7f3dcc006800 tx=0x7f3dcc006ee0).stop 2024-02-20T12:50:38.542 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:38.539+0000 7f3dd8774700 1 -- 172.21.15.64:0/3447491507 shutdown_connections 2024-02-20T12:50:38.542 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:38.539+0000 7f3dd8774700 1 --2- 172.21.15.64:0/3447491507 >> [v2:172.21.15.64:6800/1362908731,v1:172.21.15.64:6801/1362908731] conn(0x7f3db8033240 0x7f3db8035700 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:50:38.542 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:38.539+0000 7f3dd8774700 1 --2- 172.21.15.64:0/3447491507 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3dd00faed0 0x7f3dd0108af0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:50:38.542 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:38.539+0000 7f3dd8774700 1 -- 172.21.15.64:0/3447491507 >> 172.21.15.64:0/3447491507 conn(0x7f3dd00f6680 msgr2=0x7f3dd00ffc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:50:38.542 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:38.540+0000 7f3dd8774700 1 -- 172.21.15.64:0/3447491507 shutdown_connections 2024-02-20T12:50:38.542 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:38.540+0000 7f3dd8774700 1 -- 172.21.15.64:0/3447491507 wait complete. 2024-02-20T12:50:39.461 INFO:teuthology.orchestra.run.smithi064.stderr:Deploying node-exporter service with default placement... 2024-02-20T12:50:39.752 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:39 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:50:38.535258+0000 mgr.smithi064.vriyor (mgr.14116) 18 : audit [DBG] from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "target": ["mon-mgr", ""]}]: dispatch 2024-02-20T12:50:39.752 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:39 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cephadm 2024-02-20T12:50:38.535748+0000 mgr.smithi064.vriyor (mgr.14116) 19 : cephadm [INF] Saving service grafana spec with placement count:1 2024-02-20T12:50:39.752 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:39 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:50:38.538615+0000 mon.smithi064 (mon.0) 61 : audit [INF] from='mgr.14116 172.21.15.64:0/330181513' entity='mgr.smithi064.vriyor' 2024-02-20T12:50:40.210 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:40.201+0000 7f2ebc247700 1 Processor -- start 2024-02-20T12:50:40.210 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:40.201+0000 7f2ebc247700 1 -- start start 2024-02-20T12:50:40.210 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:40.201+0000 7f2ebc247700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2eb40275f0 0x7f2eb4101060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-02-20T12:50:40.210 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:40.201+0000 7f2ebc247700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2eb40fca50 con 0x7f2eb40275f0 2024-02-20T12:50:40.210 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:40.201+0000 7f2eb9fe3700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2eb40275f0 0x7f2eb4101060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:50:40.210 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:40.201+0000 7f2eb9fe3700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2eb40275f0 0x7f2eb4101060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57140/0 (socket says 172.21.15.64:57140) 2024-02-20T12:50:40.210 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:40.201+0000 7f2eb9fe3700 1 -- 172.21.15.64:0/960606326 learned_addr learned my addr 172.21.15.64:0/960606326 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-20T12:50:40.210 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:40.202+0000 7f2eb9fe3700 1 -- 172.21.15.64:0/960606326 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2eb4101550 con 0x7f2eb40275f0 2024-02-20T12:50:40.210 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:40.202+0000 7f2eb9fe3700 1 --2- 172.21.15.64:0/960606326 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2eb40275f0 0x7f2eb4101060 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0x7f2ea400b5f0 tx=0x7f2ea4006ee0).ready entity=mon.0 client_cookie=d1c7b48bd5213c24 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:50:40.210 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:40.202+0000 7f2eb8fe1700 1 -- 172.21.15.64:0/960606326 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2ea400d600 con 0x7f2eb40275f0 2024-02-20T12:50:40.210 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:40.202+0000 7f2eb8fe1700 1 -- 172.21.15.64:0/960606326 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(25 keys) v1 ==== 997+0+0 (secure 0 0 0) 0x7f2ea400dbc0 con 0x7f2eb40275f0 2024-02-20T12:50:40.210 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:40.202+0000 7f2eb8fe1700 1 -- 172.21.15.64:0/960606326 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2ea4011b90 con 0x7f2eb40275f0 2024-02-20T12:50:40.211 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:40.203+0000 7f2ebc247700 1 -- 172.21.15.64:0/960606326 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2eb40275f0 msgr2=0x7f2eb4101060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:50:40.211 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:40.203+0000 7f2ebc247700 1 --2- 172.21.15.64:0/960606326 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2eb40275f0 0x7f2eb4101060 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0x7f2ea400b5f0 tx=0x7f2ea4006ee0).stop 2024-02-20T12:50:40.211 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:40.203+0000 7f2ebc247700 1 -- 172.21.15.64:0/960606326 shutdown_connections 2024-02-20T12:50:40.211 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:40.203+0000 7f2ebc247700 1 --2- 172.21.15.64:0/960606326 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2eb40275f0 0x7f2eb4101060 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:50:40.211 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:40.203+0000 7f2ebc247700 1 -- 172.21.15.64:0/960606326 >> 172.21.15.64:0/960606326 conn(0x7f2eb40f6680 msgr2=0x7f2eb40f8aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:50:40.211 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:40.203+0000 7f2ebc247700 1 -- 172.21.15.64:0/960606326 shutdown_connections 2024-02-20T12:50:40.211 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:40.203+0000 7f2ebc247700 1 -- 172.21.15.64:0/960606326 wait complete. 2024-02-20T12:50:40.211 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:40.203+0000 7f2ebc247700 1 Processor -- start 2024-02-20T12:50:40.211 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:40.203+0000 7f2ebc247700 1 -- start start 2024-02-20T12:50:40.211 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:40.203+0000 7f2ebc247700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2eb40275f0 0x7f2eb410cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:50:40.211 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:40.203+0000 7f2ebc247700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2eb410d3f0 con 0x7f2eb40275f0 2024-02-20T12:50:40.211 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:40.204+0000 7f2eb9fe3700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2eb40275f0 0x7f2eb410cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:50:40.211 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:40.204+0000 7f2eb9fe3700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2eb40275f0 0x7f2eb410cf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57146/0 (socket says 172.21.15.64:57146) 2024-02-20T12:50:40.211 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:40.204+0000 7f2eb9fe3700 1 -- 172.21.15.64:0/4136451202 learned_addr learned my addr 172.21.15.64:0/4136451202 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-20T12:50:40.211 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:40.204+0000 7f2eb9fe3700 1 -- 172.21.15.64:0/4136451202 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ea400b200 con 0x7f2eb40275f0 2024-02-20T12:50:40.212 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:40.204+0000 7f2eb9fe3700 1 --2- 172.21.15.64:0/4136451202 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2eb40275f0 0x7f2eb410cf00 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7f2ea4006800 tx=0x7f2ea4006ee0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:50:40.212 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:40.204+0000 7f2ea27fc700 1 -- 172.21.15.64:0/4136451202 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2ea400d9c0 con 0x7f2eb40275f0 2024-02-20T12:50:40.212 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:40.204+0000 7f2ea27fc700 1 -- 172.21.15.64:0/4136451202 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(25 keys) v1 ==== 997+0+0 (secure 0 0 0) 0x7f2ea401b870 con 0x7f2eb40275f0 2024-02-20T12:50:40.212 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:40.204+0000 7f2ebc247700 1 -- 172.21.15.64:0/4136451202 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2eb410d5f0 con 0x7f2eb40275f0 2024-02-20T12:50:40.212 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:40.204+0000 7f2ea27fc700 1 -- 172.21.15.64:0/4136451202 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2ea401a740 con 0x7f2eb40275f0 2024-02-20T12:50:40.212 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:40.204+0000 7f2ebc247700 1 -- 172.21.15.64:0/4136451202 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2eb410da80 con 0x7f2eb40275f0 2024-02-20T12:50:40.212 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:40.205+0000 7f2ea27fc700 1 -- 172.21.15.64:0/4136451202 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 8) v1 ==== 37095+0+0 (secure 0 0 0) 0x7f2ea402b070 con 0x7f2eb40275f0 2024-02-20T12:50:40.212 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:40.205+0000 7f2ea27fc700 1 --2- 172.21.15.64:0/4136451202 >> [v2:172.21.15.64:6800/1362908731,v1:172.21.15.64:6801/1362908731] conn(0x7f2e9c030fa0 0x7f2e9c033460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:50:40.212 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:40.205+0000 7f2ea27fc700 1 -- 172.21.15.64:0/4136451202 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 893+0+0 (secure 0 0 0) 0x7f2ea4042c40 con 0x7f2eb40275f0 2024-02-20T12:50:40.212 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:40.205+0000 7f2eb97e2700 1 --2- 172.21.15.64:0/4136451202 >> [v2:172.21.15.64:6800/1362908731,v1:172.21.15.64:6801/1362908731] conn(0x7f2e9c030fa0 0x7f2e9c033460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:50:40.212 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:40.206+0000 7f2eb97e2700 1 --2- 172.21.15.64:0/4136451202 >> [v2:172.21.15.64:6800/1362908731,v1:172.21.15.64:6801/1362908731] conn(0x7f2e9c030fa0 0x7f2e9c033460 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 rx=0x7f2eb00096a0 tx=0x7f2eb0006b40).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:50:40.212 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:40.206+0000 7f2ebc247700 1 -- 172.21.15.64:0/4136451202 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2eb4045bb0 con 0x7f2eb40275f0 2024-02-20T12:50:40.213 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:40.211+0000 7f2ea27fc700 1 -- 172.21.15.64:0/4136451202 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+153237 (secure 0 0 0) 0x7f2ea4016030 con 0x7f2eb40275f0 2024-02-20T12:50:40.381 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:40.380+0000 7f2ebc247700 1 -- 172.21.15.64:0/4136451202 --> [v2:172.21.15.64:6800/1362908731,v1:172.21.15.64:6801/1362908731] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "node-exporter", "target": ["mon-mgr", ""]}) v1 -- 0x7f2eb4106f00 con 0x7f2e9c030fa0 2024-02-20T12:50:40.386 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:40.385+0000 7f2ea27fc700 1 -- 172.21.15.64:0/4136451202 <== mgr.14116 v2:172.21.15.64:6800/1362908731 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+34 (secure 0 0 0) 0x7f2eb4106f00 con 0x7f2e9c030fa0 2024-02-20T12:50:40.386 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: Scheduled node-exporter update... 2024-02-20T12:50:40.388 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:40.387+0000 7f2ebc247700 1 -- 172.21.15.64:0/4136451202 >> [v2:172.21.15.64:6800/1362908731,v1:172.21.15.64:6801/1362908731] conn(0x7f2e9c030fa0 msgr2=0x7f2e9c033460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:50:40.389 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:40.387+0000 7f2ebc247700 1 --2- 172.21.15.64:0/4136451202 >> [v2:172.21.15.64:6800/1362908731,v1:172.21.15.64:6801/1362908731] conn(0x7f2e9c030fa0 0x7f2e9c033460 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 rx=0x7f2eb00096a0 tx=0x7f2eb0006b40).stop 2024-02-20T12:50:40.389 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:40.387+0000 7f2ebc247700 1 -- 172.21.15.64:0/4136451202 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2eb40275f0 msgr2=0x7f2eb410cf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:50:40.389 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:40.387+0000 7f2ebc247700 1 --2- 172.21.15.64:0/4136451202 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2eb40275f0 0x7f2eb410cf00 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7f2ea4006800 tx=0x7f2ea4006ee0).stop 2024-02-20T12:50:40.389 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:40.387+0000 7f2ebc247700 1 -- 172.21.15.64:0/4136451202 shutdown_connections 2024-02-20T12:50:40.389 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:40.387+0000 7f2ebc247700 1 --2- 172.21.15.64:0/4136451202 >> [v2:172.21.15.64:6800/1362908731,v1:172.21.15.64:6801/1362908731] conn(0x7f2e9c030fa0 0x7f2e9c033460 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:50:40.389 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:40.387+0000 7f2ebc247700 1 --2- 172.21.15.64:0/4136451202 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2eb40275f0 0x7f2eb410cf00 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:50:40.389 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:40.387+0000 7f2ebc247700 1 -- 172.21.15.64:0/4136451202 >> 172.21.15.64:0/4136451202 conn(0x7f2eb40f6680 msgr2=0x7f2eb40f8120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:50:40.389 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:40.387+0000 7f2ebc247700 1 -- 172.21.15.64:0/4136451202 shutdown_connections 2024-02-20T12:50:40.389 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:40.387+0000 7f2ebc247700 1 -- 172.21.15.64:0/4136451202 wait complete. 2024-02-20T12:50:41.314 INFO:teuthology.orchestra.run.smithi064.stderr:Deploying alertmanager service with default placement... 2024-02-20T12:50:41.570 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:41 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:50:40.382366+0000 mgr.smithi064.vriyor (mgr.14116) 20 : audit [DBG] from='client.14146 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "target": ["mon-mgr", ""]}]: dispatch 2024-02-20T12:50:41.571 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:41 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cephadm 2024-02-20T12:50:40.382880+0000 mgr.smithi064.vriyor (mgr.14116) 21 : cephadm [INF] Saving service node-exporter spec with placement * 2024-02-20T12:50:41.571 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:41 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:50:40.385912+0000 mon.smithi064 (mon.0) 62 : audit [INF] from='mgr.14116 172.21.15.64:0/330181513' entity='mgr.smithi064.vriyor' 2024-02-20T12:50:42.331 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:41.964+0000 7fe217703700 1 Processor -- start 2024-02-20T12:50:42.331 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:41.964+0000 7fe217703700 1 -- start start 2024-02-20T12:50:42.331 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:41.965+0000 7fe217703700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe2100fc5a0 0x7fe2100fc9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-02-20T12:50:42.331 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:41.965+0000 7fe217703700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2100fce90 con 0x7fe2100fc5a0 2024-02-20T12:50:42.331 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:41.965+0000 7fe21549f700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe2100fc5a0 0x7fe2100fc9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:50:42.331 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:41.965+0000 7fe21549f700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe2100fc5a0 0x7fe2100fc9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42658/0 (socket says 172.21.15.64:42658) 2024-02-20T12:50:42.332 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:41.965+0000 7fe21549f700 1 -- 172.21.15.64:0/2983683771 learned_addr learned my addr 172.21.15.64:0/2983683771 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-20T12:50:42.332 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:41.965+0000 7fe21549f700 1 -- 172.21.15.64:0/2983683771 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2100fcfd0 con 0x7fe2100fc5a0 2024-02-20T12:50:42.332 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:41.965+0000 7fe21549f700 1 --2- 172.21.15.64:0/2983683771 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe2100fc5a0 0x7fe2100fc9a0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7fe20c00b5f0 tx=0x7fe20c006ee0).ready entity=mon.0 client_cookie=faed7cb28f79369f server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:50:42.332 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:41.965+0000 7fe203fff700 1 -- 172.21.15.64:0/2983683771 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe20c00d600 con 0x7fe2100fc5a0 2024-02-20T12:50:42.332 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:41.965+0000 7fe203fff700 1 -- 172.21.15.64:0/2983683771 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(25 keys) v1 ==== 997+0+0 (secure 0 0 0) 0x7fe20c00dbc0 con 0x7fe2100fc5a0 2024-02-20T12:50:42.332 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:41.965+0000 7fe203fff700 1 -- 172.21.15.64:0/2983683771 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe20c011b90 con 0x7fe2100fc5a0 2024-02-20T12:50:42.332 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:41.966+0000 7fe217703700 1 -- 172.21.15.64:0/2983683771 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe2100fc5a0 msgr2=0x7fe2100fc9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:50:42.332 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:41.966+0000 7fe217703700 1 --2- 172.21.15.64:0/2983683771 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe2100fc5a0 0x7fe2100fc9a0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7fe20c00b5f0 tx=0x7fe20c006ee0).stop 2024-02-20T12:50:42.332 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:41.966+0000 7fe217703700 1 -- 172.21.15.64:0/2983683771 shutdown_connections 2024-02-20T12:50:42.332 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:41.966+0000 7fe217703700 1 --2- 172.21.15.64:0/2983683771 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe2100fc5a0 0x7fe2100fc9a0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:50:42.332 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:41.966+0000 7fe217703700 1 -- 172.21.15.64:0/2983683771 >> 172.21.15.64:0/2983683771 conn(0x7fe2100f5df0 msgr2=0x7fe2100f8210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:50:42.332 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:41.966+0000 7fe217703700 1 -- 172.21.15.64:0/2983683771 shutdown_connections 2024-02-20T12:50:42.332 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:41.966+0000 7fe217703700 1 -- 172.21.15.64:0/2983683771 wait complete. 2024-02-20T12:50:42.332 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:41.967+0000 7fe217703700 1 Processor -- start 2024-02-20T12:50:42.332 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:41.967+0000 7fe217703700 1 -- start start 2024-02-20T12:50:42.333 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:41.967+0000 7fe217703700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe2100fc5a0 0x7fe21010cf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:50:42.333 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:41.967+0000 7fe217703700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe21010d400 con 0x7fe2100fc5a0 2024-02-20T12:50:42.333 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:41.967+0000 7fe21549f700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe2100fc5a0 0x7fe21010cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:50:42.333 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:41.967+0000 7fe21549f700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe2100fc5a0 0x7fe21010cf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42666/0 (socket says 172.21.15.64:42666) 2024-02-20T12:50:42.333 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:41.967+0000 7fe21549f700 1 -- 172.21.15.64:0/3443138043 learned_addr learned my addr 172.21.15.64:0/3443138043 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-20T12:50:42.333 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:41.967+0000 7fe21549f700 1 -- 172.21.15.64:0/3443138043 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe20c00b200 con 0x7fe2100fc5a0 2024-02-20T12:50:42.333 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:41.967+0000 7fe21549f700 1 --2- 172.21.15.64:0/3443138043 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe2100fc5a0 0x7fe21010cf10 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7fe20c006800 tx=0x7fe20c006ee0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:50:42.333 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:41.968+0000 7fe201ffb700 1 -- 172.21.15.64:0/3443138043 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe20c00d9c0 con 0x7fe2100fc5a0 2024-02-20T12:50:42.333 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:41.968+0000 7fe201ffb700 1 -- 172.21.15.64:0/3443138043 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(25 keys) v1 ==== 997+0+0 (secure 0 0 0) 0x7fe20c01b870 con 0x7fe2100fc5a0 2024-02-20T12:50:42.334 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:41.968+0000 7fe217703700 1 -- 172.21.15.64:0/3443138043 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe21010d600 con 0x7fe2100fc5a0 2024-02-20T12:50:42.334 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:41.968+0000 7fe201ffb700 1 -- 172.21.15.64:0/3443138043 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe20c01a740 con 0x7fe2100fc5a0 2024-02-20T12:50:42.334 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:41.968+0000 7fe217703700 1 -- 172.21.15.64:0/3443138043 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe21010da90 con 0x7fe2100fc5a0 2024-02-20T12:50:42.334 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:41.969+0000 7fe201ffb700 1 -- 172.21.15.64:0/3443138043 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 8) v1 ==== 37095+0+0 (secure 0 0 0) 0x7fe20c02b070 con 0x7fe2100fc5a0 2024-02-20T12:50:42.334 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:41.969+0000 7fe201ffb700 1 --2- 172.21.15.64:0/3443138043 >> [v2:172.21.15.64:6800/1362908731,v1:172.21.15.64:6801/1362908731] conn(0x7fe1f8030ff0 0x7fe1f80334b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:50:42.334 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:41.969+0000 7fe217703700 1 -- 172.21.15.64:0/3443138043 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe210045bb0 con 0x7fe2100fc5a0 2024-02-20T12:50:42.335 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:41.969+0000 7fe201ffb700 1 -- 172.21.15.64:0/3443138043 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 893+0+0 (secure 0 0 0) 0x7fe20c042c20 con 0x7fe2100fc5a0 2024-02-20T12:50:42.335 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:41.969+0000 7fe214c9e700 1 --2- 172.21.15.64:0/3443138043 >> [v2:172.21.15.64:6800/1362908731,v1:172.21.15.64:6801/1362908731] conn(0x7fe1f8030ff0 0x7fe1f80334b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:50:42.335 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:41.969+0000 7fe214c9e700 1 --2- 172.21.15.64:0/3443138043 >> [v2:172.21.15.64:6800/1362908731,v1:172.21.15.64:6801/1362908731] conn(0x7fe1f8030ff0 0x7fe1f80334b0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 rx=0x7fe2040096a0 tx=0x7fe204006b40).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:50:42.335 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:41.973+0000 7fe201ffb700 1 -- 172.21.15.64:0/3443138043 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+153237 (secure 0 0 0) 0x7fe20c016070 con 0x7fe2100fc5a0 2024-02-20T12:50:42.335 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:42.143+0000 7fe217703700 1 -- 172.21.15.64:0/3443138043 --> [v2:172.21.15.64:6800/1362908731,v1:172.21.15.64:6801/1362908731] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "alertmanager", "target": ["mon-mgr", ""]}) v1 -- 0x7fe21010e2f0 con 0x7fe1f8030ff0 2024-02-20T12:50:42.335 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:42.147+0000 7fe201ffb700 1 -- 172.21.15.64:0/3443138043 <== mgr.14116 v2:172.21.15.64:6800/1362908731 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+33 (secure 0 0 0) 0x7fe21010e2f0 con 0x7fe1f8030ff0 2024-02-20T12:50:42.335 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:42.149+0000 7fe217703700 1 -- 172.21.15.64:0/3443138043 >> [v2:172.21.15.64:6800/1362908731,v1:172.21.15.64:6801/1362908731] conn(0x7fe1f8030ff0 msgr2=0x7fe1f80334b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:50:42.335 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:42.149+0000 7fe217703700 1 --2- 172.21.15.64:0/3443138043 >> [v2:172.21.15.64:6800/1362908731,v1:172.21.15.64:6801/1362908731] conn(0x7fe1f8030ff0 0x7fe1f80334b0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 rx=0x7fe2040096a0 tx=0x7fe204006b40).stop 2024-02-20T12:50:42.335 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:42.149+0000 7fe217703700 1 -- 172.21.15.64:0/3443138043 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe2100fc5a0 msgr2=0x7fe21010cf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:50:42.335 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:42.149+0000 7fe217703700 1 --2- 172.21.15.64:0/3443138043 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe2100fc5a0 0x7fe21010cf10 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7fe20c006800 tx=0x7fe20c006ee0).stop 2024-02-20T12:50:42.335 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:42.149+0000 7fe217703700 1 -- 172.21.15.64:0/3443138043 shutdown_connections 2024-02-20T12:50:42.335 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:42.149+0000 7fe217703700 1 --2- 172.21.15.64:0/3443138043 >> [v2:172.21.15.64:6800/1362908731,v1:172.21.15.64:6801/1362908731] conn(0x7fe1f8030ff0 0x7fe1f80334b0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:50:42.335 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:42.149+0000 7fe217703700 1 --2- 172.21.15.64:0/3443138043 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe2100fc5a0 0x7fe21010cf10 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:50:42.335 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:42.149+0000 7fe217703700 1 -- 172.21.15.64:0/3443138043 >> 172.21.15.64:0/3443138043 conn(0x7fe2100f5df0 msgr2=0x7fe2100f78b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:50:42.336 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:42.149+0000 7fe217703700 1 -- 172.21.15.64:0/3443138043 shutdown_connections 2024-02-20T12:50:42.336 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:42.149+0000 7fe217703700 1 -- 172.21.15.64:0/3443138043 wait complete. 2024-02-20T12:50:42.336 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: Scheduled alertmanager update... 2024-02-20T12:50:43.221 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:43 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:50:42.145462+0000 mgr.smithi064.vriyor (mgr.14116) 22 : audit [DBG] from='client.14148 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "target": ["mon-mgr", ""]}]: dispatch 2024-02-20T12:50:43.221 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:43 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cephadm 2024-02-20T12:50:42.145991+0000 mgr.smithi064.vriyor (mgr.14116) 23 : cephadm [INF] Saving service alertmanager spec with placement count:1 2024-02-20T12:50:43.221 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:43 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:50:42.148267+0000 mon.smithi064 (mon.0) 63 : audit [INF] from='mgr.14116 172.21.15.64:0/330181513' entity='mgr.smithi064.vriyor' 2024-02-20T12:50:43.221 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:43 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:50:42.194299+0000 mon.smithi064 (mon.0) 64 : audit [INF] from='mgr.14116 172.21.15.64:0/330181513' entity='mgr.smithi064.vriyor' 2024-02-20T12:50:43.221 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:43 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:50:42.428329+0000 mgr.smithi064.vriyor (mgr.14116) 24 : cluster [DBG] pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:50:43.222 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:43 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:50:42.603250+0000 mon.smithi064 (mon.0) 65 : audit [INF] from='mgr.14116 172.21.15.64:0/330181513' entity='mgr.smithi064.vriyor' 2024-02-20T12:50:43.556 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:43.553+0000 7f11a3b64700 1 Processor -- start 2024-02-20T12:50:43.556 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:43.553+0000 7f11a3b64700 1 -- start start 2024-02-20T12:50:43.556 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:43.553+0000 7f11a3b64700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f119c027630 0x7f119c0fec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-02-20T12:50:43.556 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:43.553+0000 7f11a3b64700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f119c0ff110 con 0x7f119c027630 2024-02-20T12:50:43.556 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:43.554+0000 7f11a1900700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f119c027630 0x7f119c0fec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:50:43.556 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:43.554+0000 7f11a1900700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f119c027630 0x7f119c0fec20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42682/0 (socket says 172.21.15.64:42682) 2024-02-20T12:50:43.556 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:43.554+0000 7f11a1900700 1 -- 172.21.15.64:0/1365248946 learned_addr learned my addr 172.21.15.64:0/1365248946 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-20T12:50:43.556 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:43.554+0000 7f11a1900700 1 -- 172.21.15.64:0/1365248946 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f119c0ff250 con 0x7f119c027630 2024-02-20T12:50:43.556 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:43.554+0000 7f11a1900700 1 --2- 172.21.15.64:0/1365248946 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f119c027630 0x7f119c0fec20 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7f119800b5f0 tx=0x7f1198006ee0).ready entity=mon.0 client_cookie=c6109139bb1dc2c6 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:50:43.556 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:43.554+0000 7f11a08fe700 1 -- 172.21.15.64:0/1365248946 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f119800d600 con 0x7f119c027630 2024-02-20T12:50:43.557 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:43.554+0000 7f11a08fe700 1 -- 172.21.15.64:0/1365248946 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(25 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f119800dbf0 con 0x7f119c027630 2024-02-20T12:50:43.557 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:43.554+0000 7f11a08fe700 1 -- 172.21.15.64:0/1365248946 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1198011bf0 con 0x7f119c027630 2024-02-20T12:50:43.557 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:43.555+0000 7f11a3b64700 1 -- 172.21.15.64:0/1365248946 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f119c027630 msgr2=0x7f119c0fec20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:50:43.557 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:43.555+0000 7f11a3b64700 1 --2- 172.21.15.64:0/1365248946 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f119c027630 0x7f119c0fec20 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7f119800b5f0 tx=0x7f1198006ee0).stop 2024-02-20T12:50:43.558 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:43.555+0000 7f11a3b64700 1 -- 172.21.15.64:0/1365248946 shutdown_connections 2024-02-20T12:50:43.558 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:43.555+0000 7f11a3b64700 1 --2- 172.21.15.64:0/1365248946 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f119c027630 0x7f119c0fec20 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:50:43.558 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:43.555+0000 7f11a3b64700 1 -- 172.21.15.64:0/1365248946 >> 172.21.15.64:0/1365248946 conn(0x7f119c0f6680 msgr2=0x7f119c0f8aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:50:43.558 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:43.555+0000 7f11a3b64700 1 -- 172.21.15.64:0/1365248946 shutdown_connections 2024-02-20T12:50:43.558 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:43.555+0000 7f11a3b64700 1 -- 172.21.15.64:0/1365248946 wait complete. 2024-02-20T12:50:43.558 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:43.556+0000 7f11a3b64700 1 Processor -- start 2024-02-20T12:50:43.559 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:43.556+0000 7f11a3b64700 1 -- start start 2024-02-20T12:50:43.559 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:43.556+0000 7f11a3b64700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f119c027630 0x7f119c10ad90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:50:43.559 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:43.556+0000 7f11a3b64700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f119c10b280 con 0x7f119c027630 2024-02-20T12:50:43.559 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:43.556+0000 7f11a1900700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f119c027630 0x7f119c10ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:50:43.559 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:43.556+0000 7f11a1900700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f119c027630 0x7f119c10ad90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42686/0 (socket says 172.21.15.64:42686) 2024-02-20T12:50:43.559 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:43.556+0000 7f11a1900700 1 -- 172.21.15.64:0/2973576987 learned_addr learned my addr 172.21.15.64:0/2973576987 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-20T12:50:43.559 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:43.556+0000 7f11a1900700 1 -- 172.21.15.64:0/2973576987 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f119800b200 con 0x7f119c027630 2024-02-20T12:50:43.560 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:43.557+0000 7f11a1900700 1 --2- 172.21.15.64:0/2973576987 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f119c027630 0x7f119c10ad90 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7f1198006800 tx=0x7f1198006ee0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:50:43.560 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:43.557+0000 7f118a7fc700 1 -- 172.21.15.64:0/2973576987 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f119800dec0 con 0x7f119c027630 2024-02-20T12:50:43.560 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:43.557+0000 7f11a3b64700 1 -- 172.21.15.64:0/2973576987 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f119c10b480 con 0x7f119c027630 2024-02-20T12:50:43.560 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:43.557+0000 7f118a7fc700 1 -- 172.21.15.64:0/2973576987 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(25 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f119801b8a0 con 0x7f119c027630 2024-02-20T12:50:43.560 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:43.557+0000 7f118a7fc700 1 -- 172.21.15.64:0/2973576987 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f119801a7b0 con 0x7f119c027630 2024-02-20T12:50:43.560 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:43.557+0000 7f11a3b64700 1 -- 172.21.15.64:0/2973576987 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f119c10c0c0 con 0x7f119c027630 2024-02-20T12:50:43.560 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:43.558+0000 7f118a7fc700 1 -- 172.21.15.64:0/2973576987 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 8) v1 ==== 37095+0+0 (secure 0 0 0) 0x7f119802b070 con 0x7f119c027630 2024-02-20T12:50:43.560 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:43.558+0000 7f118a7fc700 1 --2- 172.21.15.64:0/2973576987 >> [v2:172.21.15.64:6800/1362908731,v1:172.21.15.64:6801/1362908731] conn(0x7f1184030fa0 0x7f1184033460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:50:43.560 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:43.558+0000 7f118a7fc700 1 -- 172.21.15.64:0/2973576987 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 893+0+0 (secure 0 0 0) 0x7f1198043620 con 0x7f119c027630 2024-02-20T12:50:43.560 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:43.558+0000 7f11a3b64700 1 -- 172.21.15.64:0/2973576987 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f119c10b910 con 0x7f119c027630 2024-02-20T12:50:43.561 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:43.559+0000 7f11a10ff700 1 --2- 172.21.15.64:0/2973576987 >> [v2:172.21.15.64:6800/1362908731,v1:172.21.15.64:6801/1362908731] conn(0x7f1184030fa0 0x7f1184033460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:50:43.561 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:43.559+0000 7f11a10ff700 1 --2- 172.21.15.64:0/2973576987 >> [v2:172.21.15.64:6800/1362908731,v1:172.21.15.64:6801/1362908731] conn(0x7f1184030fa0 0x7f1184033460 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 rx=0x7f118c0096a0 tx=0x7f118c006b40).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:50:43.564 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:43.563+0000 7f118a7fc700 1 -- 172.21.15.64:0/2973576987 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+153237 (secure 0 0 0) 0x7f1198016070 con 0x7f119c027630 2024-02-20T12:50:43.765 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:43.764+0000 7f11a3b64700 1 -- 172.21.15.64:0/2973576987 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) v1 -- 0x7f119c0591e0 con 0x7f119c027630 2024-02-20T12:50:43.768 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:43.767+0000 7f118a7fc700 1 -- 172.21.15.64:0/2973576987 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v9)=0 v9) v1 ==== 142+0+0 (secure 0 0 0) 0x7f119802b8f0 con 0x7f119c027630 2024-02-20T12:50:43.771 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:43.769+0000 7f11a3b64700 1 -- 172.21.15.64:0/2973576987 >> [v2:172.21.15.64:6800/1362908731,v1:172.21.15.64:6801/1362908731] conn(0x7f1184030fa0 msgr2=0x7f1184033460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:50:43.771 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:43.769+0000 7f11a3b64700 1 --2- 172.21.15.64:0/2973576987 >> [v2:172.21.15.64:6800/1362908731,v1:172.21.15.64:6801/1362908731] conn(0x7f1184030fa0 0x7f1184033460 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 rx=0x7f118c0096a0 tx=0x7f118c006b40).stop 2024-02-20T12:50:43.771 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:43.769+0000 7f11a3b64700 1 -- 172.21.15.64:0/2973576987 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f119c027630 msgr2=0x7f119c10ad90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:50:43.771 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:43.769+0000 7f11a3b64700 1 --2- 172.21.15.64:0/2973576987 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f119c027630 0x7f119c10ad90 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7f1198006800 tx=0x7f1198006ee0).stop 2024-02-20T12:50:43.771 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:43.769+0000 7f11a3b64700 1 -- 172.21.15.64:0/2973576987 shutdown_connections 2024-02-20T12:50:43.771 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:43.769+0000 7f11a3b64700 1 --2- 172.21.15.64:0/2973576987 >> [v2:172.21.15.64:6800/1362908731,v1:172.21.15.64:6801/1362908731] conn(0x7f1184030fa0 0x7f1184033460 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:50:43.772 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:43.769+0000 7f11a3b64700 1 --2- 172.21.15.64:0/2973576987 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f119c027630 0x7f119c10ad90 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:50:43.772 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:43.769+0000 7f11a3b64700 1 -- 172.21.15.64:0/2973576987 >> 172.21.15.64:0/2973576987 conn(0x7f119c0f6680 msgr2=0x7f119c0f81c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:50:43.772 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:43.769+0000 7f11a3b64700 1 -- 172.21.15.64:0/2973576987 shutdown_connections 2024-02-20T12:50:43.772 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:43.769+0000 7f11a3b64700 1 -- 172.21.15.64:0/2973576987 wait complete. 2024-02-20T12:50:45.019 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:44 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:50:43.768385+0000 mon.smithi064 (mon.0) 66 : audit [INF] from='client.? 172.21.15.64:0/2973576987' entity='client.admin' 2024-02-20T12:50:45.019 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:44 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:50:43.792286+0000 mon.smithi064 (mon.0) 67 : audit [INF] from='mgr.14116 172.21.15.64:0/330181513' entity='mgr.smithi064.vriyor' 2024-02-20T12:50:46.062 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:45 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:50:44.428616+0000 mgr.smithi064.vriyor (mgr.14116) 25 : cluster [DBG] pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:50:46.127 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:46.124+0000 7f1e8f26d700 1 Processor -- start 2024-02-20T12:50:46.127 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:46.125+0000 7f1e8f26d700 1 -- start start 2024-02-20T12:50:46.127 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:46.125+0000 7f1e8f26d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1e8806ade0 0x7f1e880febe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-02-20T12:50:46.127 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:46.125+0000 7f1e8f26d700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e880ff0d0 con 0x7f1e8806ade0 2024-02-20T12:50:46.127 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:46.125+0000 7f1e8d009700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1e8806ade0 0x7f1e880febe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:50:46.127 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:46.125+0000 7f1e8d009700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1e8806ade0 0x7f1e880febe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42698/0 (socket says 172.21.15.64:42698) 2024-02-20T12:50:46.128 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:46.125+0000 7f1e8d009700 1 -- 172.21.15.64:0/1298456399 learned_addr learned my addr 172.21.15.64:0/1298456399 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-20T12:50:46.128 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:46.125+0000 7f1e8d009700 1 -- 172.21.15.64:0/1298456399 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e880ff210 con 0x7f1e8806ade0 2024-02-20T12:50:46.128 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:46.125+0000 7f1e8d009700 1 --2- 172.21.15.64:0/1298456399 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1e8806ade0 0x7f1e880febe0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0x7f1e8400b5f0 tx=0x7f1e84006ee0).ready entity=mon.0 client_cookie=437d03337ac79336 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:50:46.128 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:46.126+0000 7f1e77fff700 1 -- 172.21.15.64:0/1298456399 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1e8400d600 con 0x7f1e8806ade0 2024-02-20T12:50:46.128 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:46.126+0000 7f1e77fff700 1 -- 172.21.15.64:0/1298456399 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(25 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f1e8400dbf0 con 0x7f1e8806ade0 2024-02-20T12:50:46.128 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:46.126+0000 7f1e77fff700 1 -- 172.21.15.64:0/1298456399 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1e84011bf0 con 0x7f1e8806ade0 2024-02-20T12:50:46.128 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:46.126+0000 7f1e8f26d700 1 -- 172.21.15.64:0/1298456399 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1e8806ade0 msgr2=0x7f1e880febe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:50:46.128 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:46.126+0000 7f1e8f26d700 1 --2- 172.21.15.64:0/1298456399 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1e8806ade0 0x7f1e880febe0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0x7f1e8400b5f0 tx=0x7f1e84006ee0).stop 2024-02-20T12:50:46.128 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:46.126+0000 7f1e8f26d700 1 -- 172.21.15.64:0/1298456399 shutdown_connections 2024-02-20T12:50:46.130 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:46.126+0000 7f1e8f26d700 1 --2- 172.21.15.64:0/1298456399 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1e8806ade0 0x7f1e880febe0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:50:46.130 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:46.126+0000 7f1e8f26d700 1 -- 172.21.15.64:0/1298456399 >> 172.21.15.64:0/1298456399 conn(0x7f1e880f6680 msgr2=0x7f1e880f8aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:50:46.130 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:46.126+0000 7f1e8f26d700 1 -- 172.21.15.64:0/1298456399 shutdown_connections 2024-02-20T12:50:46.130 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:46.126+0000 7f1e8f26d700 1 -- 172.21.15.64:0/1298456399 wait complete. 2024-02-20T12:50:46.130 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:46.127+0000 7f1e8f26d700 1 Processor -- start 2024-02-20T12:50:46.130 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:46.127+0000 7f1e8f26d700 1 -- start start 2024-02-20T12:50:46.130 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:46.127+0000 7f1e8f26d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1e8806ade0 0x7f1e8810ada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:50:46.131 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:46.127+0000 7f1e8f26d700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e8810b290 con 0x7f1e8806ade0 2024-02-20T12:50:46.131 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:46.127+0000 7f1e8d009700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1e8806ade0 0x7f1e8810ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:50:46.131 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:46.127+0000 7f1e8d009700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1e8806ade0 0x7f1e8810ada0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42712/0 (socket says 172.21.15.64:42712) 2024-02-20T12:50:46.131 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:46.127+0000 7f1e8d009700 1 -- 172.21.15.64:0/2553284376 learned_addr learned my addr 172.21.15.64:0/2553284376 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-20T12:50:46.131 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:46.128+0000 7f1e8d009700 1 -- 172.21.15.64:0/2553284376 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e8400b200 con 0x7f1e8806ade0 2024-02-20T12:50:46.131 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:46.128+0000 7f1e8d009700 1 --2- 172.21.15.64:0/2553284376 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1e8806ade0 0x7f1e8810ada0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 rx=0x7f1e84011de0 tx=0x7f1e84006ee0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:50:46.131 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:46.128+0000 7f1e75ffb700 1 -- 172.21.15.64:0/2553284376 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1e8401b410 con 0x7f1e8806ade0 2024-02-20T12:50:46.131 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:46.128+0000 7f1e75ffb700 1 -- 172.21.15.64:0/2553284376 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(25 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f1e8401ba00 con 0x7f1e8806ade0 2024-02-20T12:50:46.131 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:46.128+0000 7f1e8f26d700 1 -- 172.21.15.64:0/2553284376 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e8810b490 con 0x7f1e8806ade0 2024-02-20T12:50:46.131 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:46.128+0000 7f1e75ffb700 1 -- 172.21.15.64:0/2553284376 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1e8401a770 con 0x7f1e8806ade0 2024-02-20T12:50:46.131 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:46.128+0000 7f1e8f26d700 1 -- 172.21.15.64:0/2553284376 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e8810bfd0 con 0x7f1e8806ade0 2024-02-20T12:50:46.131 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:46.129+0000 7f1e8f26d700 1 -- 172.21.15.64:0/2553284376 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e880591e0 con 0x7f1e8806ade0 2024-02-20T12:50:46.132 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:46.129+0000 7f1e75ffb700 1 -- 172.21.15.64:0/2553284376 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 8) v1 ==== 37095+0+0 (secure 0 0 0) 0x7f1e8402b070 con 0x7f1e8806ade0 2024-02-20T12:50:46.132 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:46.130+0000 7f1e75ffb700 1 --2- 172.21.15.64:0/2553284376 >> [v2:172.21.15.64:6800/1362908731,v1:172.21.15.64:6801/1362908731] conn(0x7f1e70030ff0 0x7f1e700334b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:50:46.132 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:46.130+0000 7f1e75ffb700 1 -- 172.21.15.64:0/2553284376 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 893+0+0 (secure 0 0 0) 0x7f1e84043190 con 0x7f1e8806ade0 2024-02-20T12:50:46.133 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:46.130+0000 7f1e8c808700 1 --2- 172.21.15.64:0/2553284376 >> [v2:172.21.15.64:6800/1362908731,v1:172.21.15.64:6801/1362908731] conn(0x7f1e70030ff0 0x7f1e700334b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:50:46.133 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:46.130+0000 7f1e8c808700 1 --2- 172.21.15.64:0/2553284376 >> [v2:172.21.15.64:6800/1362908731,v1:172.21.15.64:6801/1362908731] conn(0x7f1e70030ff0 0x7f1e700334b0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 rx=0x7f1e780096a0 tx=0x7f1e78006b40).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:50:46.135 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:46.134+0000 7f1e75ffb700 1 -- 172.21.15.64:0/2553284376 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+153237 (secure 0 0 0) 0x7f1e8401bb70 con 0x7f1e8806ade0 2024-02-20T12:50:46.295 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:46.294+0000 7f1e8f26d700 1 -- 172.21.15.64:0/2553284376 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) v1 -- 0x7f1e8810ce00 con 0x7f1e8806ade0 2024-02-20T12:50:46.298 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:46.297+0000 7f1e75ffb700 1 -- 172.21.15.64:0/2553284376 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v10)=0 v10) v1 ==== 130+0+0 (secure 0 0 0) 0x7f1e84016020 con 0x7f1e8806ade0 2024-02-20T12:50:46.301 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:46.299+0000 7f1e8f26d700 1 -- 172.21.15.64:0/2553284376 >> [v2:172.21.15.64:6800/1362908731,v1:172.21.15.64:6801/1362908731] conn(0x7f1e70030ff0 msgr2=0x7f1e700334b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:50:46.301 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:46.299+0000 7f1e8f26d700 1 --2- 172.21.15.64:0/2553284376 >> [v2:172.21.15.64:6800/1362908731,v1:172.21.15.64:6801/1362908731] conn(0x7f1e70030ff0 0x7f1e700334b0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 rx=0x7f1e780096a0 tx=0x7f1e78006b40).stop 2024-02-20T12:50:46.301 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:46.299+0000 7f1e8f26d700 1 -- 172.21.15.64:0/2553284376 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1e8806ade0 msgr2=0x7f1e8810ada0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:50:46.301 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:46.299+0000 7f1e8f26d700 1 --2- 172.21.15.64:0/2553284376 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1e8806ade0 0x7f1e8810ada0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 rx=0x7f1e84011de0 tx=0x7f1e84006ee0).stop 2024-02-20T12:50:46.301 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:46.299+0000 7f1e8f26d700 1 -- 172.21.15.64:0/2553284376 shutdown_connections 2024-02-20T12:50:46.301 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:46.299+0000 7f1e8f26d700 1 --2- 172.21.15.64:0/2553284376 >> [v2:172.21.15.64:6800/1362908731,v1:172.21.15.64:6801/1362908731] conn(0x7f1e70030ff0 0x7f1e700334b0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:50:46.301 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:46.299+0000 7f1e8f26d700 1 --2- 172.21.15.64:0/2553284376 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1e8806ade0 0x7f1e8810ada0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:50:46.301 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:46.299+0000 7f1e8f26d700 1 -- 172.21.15.64:0/2553284376 >> 172.21.15.64:0/2553284376 conn(0x7f1e880f6680 msgr2=0x7f1e880f8150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:50:46.301 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:46.299+0000 7f1e8f26d700 1 -- 172.21.15.64:0/2553284376 shutdown_connections 2024-02-20T12:50:46.301 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:46.299+0000 7f1e8f26d700 1 -- 172.21.15.64:0/2553284376 wait complete. 2024-02-20T12:50:47.026 INFO:teuthology.orchestra.run.smithi064.stderr:Enabling the dashboard module... 2024-02-20T12:50:47.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:47 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:50:46.298298+0000 mon.smithi064 (mon.0) 68 : audit [INF] from='client.? 172.21.15.64:0/2553284376' entity='client.admin' 2024-02-20T12:50:47.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:47 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:50:46.429092+0000 mgr.smithi064.vriyor (mgr.14116) 26 : cluster [DBG] pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:50:47.685 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:47.654+0000 7fb20a90e700 1 Processor -- start 2024-02-20T12:50:47.685 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:47.654+0000 7fb20a90e700 1 -- start start 2024-02-20T12:50:47.685 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:47.655+0000 7fb20a90e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb20406ade0 0x7fb2040febe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-02-20T12:50:47.685 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:47.655+0000 7fb20a90e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2040ff0d0 con 0x7fb20406ade0 2024-02-20T12:50:47.685 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:47.655+0000 7fb203fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb20406ade0 0x7fb2040febe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:50:47.686 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:47.655+0000 7fb203fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb20406ade0 0x7fb2040febe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42728/0 (socket says 172.21.15.64:42728) 2024-02-20T12:50:47.686 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:47.655+0000 7fb203fff700 1 -- 172.21.15.64:0/1203453343 learned_addr learned my addr 172.21.15.64:0/1203453343 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-20T12:50:47.686 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:47.655+0000 7fb203fff700 1 -- 172.21.15.64:0/1203453343 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2040ff210 con 0x7fb20406ade0 2024-02-20T12:50:47.686 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:47.655+0000 7fb203fff700 1 --2- 172.21.15.64:0/1203453343 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb20406ade0 0x7fb2040febe0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 rx=0x7fb1f400b5f0 tx=0x7fb1f4006ee0).ready entity=mon.0 client_cookie=21cd5aad5ac362fa server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:50:47.686 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:47.655+0000 7fb202ffd700 1 -- 172.21.15.64:0/1203453343 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb1f400d600 con 0x7fb20406ade0 2024-02-20T12:50:47.686 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:47.655+0000 7fb202ffd700 1 -- 172.21.15.64:0/1203453343 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(25 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7fb1f400dbf0 con 0x7fb20406ade0 2024-02-20T12:50:47.687 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:47.655+0000 7fb202ffd700 1 -- 172.21.15.64:0/1203453343 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb1f4011bf0 con 0x7fb20406ade0 2024-02-20T12:50:47.687 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:47.656+0000 7fb20a90e700 1 -- 172.21.15.64:0/1203453343 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb20406ade0 msgr2=0x7fb2040febe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:50:47.687 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:47.656+0000 7fb20a90e700 1 --2- 172.21.15.64:0/1203453343 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb20406ade0 0x7fb2040febe0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 rx=0x7fb1f400b5f0 tx=0x7fb1f4006ee0).stop 2024-02-20T12:50:47.687 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:47.656+0000 7fb20a90e700 1 -- 172.21.15.64:0/1203453343 shutdown_connections 2024-02-20T12:50:47.687 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:47.656+0000 7fb20a90e700 1 --2- 172.21.15.64:0/1203453343 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb20406ade0 0x7fb2040febe0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:50:47.687 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:47.656+0000 7fb20a90e700 1 -- 172.21.15.64:0/1203453343 >> 172.21.15.64:0/1203453343 conn(0x7fb2040f6680 msgr2=0x7fb2040f8aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:50:47.688 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:47.656+0000 7fb20a90e700 1 -- 172.21.15.64:0/1203453343 shutdown_connections 2024-02-20T12:50:47.688 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:47.656+0000 7fb20a90e700 1 -- 172.21.15.64:0/1203453343 wait complete. 2024-02-20T12:50:47.688 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:47.656+0000 7fb20a90e700 1 Processor -- start 2024-02-20T12:50:47.688 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:47.657+0000 7fb20a90e700 1 -- start start 2024-02-20T12:50:47.688 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:47.657+0000 7fb20a90e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb20406ade0 0x7fb20410ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:50:47.689 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:47.657+0000 7fb20a90e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb20410b260 con 0x7fb20406ade0 2024-02-20T12:50:47.689 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:47.657+0000 7fb203fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb20406ade0 0x7fb20410ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:50:47.689 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:47.657+0000 7fb203fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb20406ade0 0x7fb20410ad70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42744/0 (socket says 172.21.15.64:42744) 2024-02-20T12:50:47.689 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:47.657+0000 7fb203fff700 1 -- 172.21.15.64:0/3949255176 learned_addr learned my addr 172.21.15.64:0/3949255176 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-20T12:50:47.689 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:47.657+0000 7fb203fff700 1 -- 172.21.15.64:0/3949255176 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1f400b200 con 0x7fb20406ade0 2024-02-20T12:50:47.689 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:47.657+0000 7fb203fff700 1 --2- 172.21.15.64:0/3949255176 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb20406ade0 0x7fb20410ad70 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 rx=0x7fb1f4011de0 tx=0x7fb1f4006ee0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:50:47.690 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:47.658+0000 7fb200ff9700 1 -- 172.21.15.64:0/3949255176 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb1f401b410 con 0x7fb20406ade0 2024-02-20T12:50:47.690 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:47.658+0000 7fb200ff9700 1 -- 172.21.15.64:0/3949255176 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(25 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7fb1f401ba00 con 0x7fb20406ade0 2024-02-20T12:50:47.690 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:47.658+0000 7fb20a90e700 1 -- 172.21.15.64:0/3949255176 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb20410b460 con 0x7fb20406ade0 2024-02-20T12:50:47.690 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:47.658+0000 7fb200ff9700 1 -- 172.21.15.64:0/3949255176 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb1f401a770 con 0x7fb20406ade0 2024-02-20T12:50:47.690 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:47.658+0000 7fb20a90e700 1 -- 172.21.15.64:0/3949255176 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb20410c070 con 0x7fb20406ade0 2024-02-20T12:50:47.690 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:47.658+0000 7fb200ff9700 1 -- 172.21.15.64:0/3949255176 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 8) v1 ==== 37095+0+0 (secure 0 0 0) 0x7fb1f402b070 con 0x7fb20406ade0 2024-02-20T12:50:47.690 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:47.658+0000 7fb20a90e700 1 -- 172.21.15.64:0/3949255176 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2040591e0 con 0x7fb20406ade0 2024-02-20T12:50:47.691 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:47.659+0000 7fb200ff9700 1 --2- 172.21.15.64:0/3949255176 >> [v2:172.21.15.64:6800/1362908731,v1:172.21.15.64:6801/1362908731] conn(0x7fb1f0030fa0 0x7fb1f0033460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:50:47.691 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:47.659+0000 7fb200ff9700 1 -- 172.21.15.64:0/3949255176 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 893+0+0 (secure 0 0 0) 0x7fb1f4042be0 con 0x7fb20406ade0 2024-02-20T12:50:47.691 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:47.659+0000 7fb2037fe700 1 --2- 172.21.15.64:0/3949255176 >> [v2:172.21.15.64:6800/1362908731,v1:172.21.15.64:6801/1362908731] conn(0x7fb1f0030fa0 0x7fb1f0033460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:50:47.691 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:47.659+0000 7fb2037fe700 1 --2- 172.21.15.64:0/3949255176 >> [v2:172.21.15.64:6800/1362908731,v1:172.21.15.64:6801/1362908731] conn(0x7fb1f0030fa0 0x7fb1f0033460 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 rx=0x7fb1f80096a0 tx=0x7fb1f8006b40).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:50:47.691 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:47.663+0000 7fb200ff9700 1 -- 172.21.15.64:0/3949255176 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+153237 (secure 0 0 0) 0x7fb1f4006a70 con 0x7fb20406ade0 2024-02-20T12:50:47.867 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:47.866+0000 7fb20a90e700 1 -- 172.21.15.64:0/3949255176 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) v1 -- 0x7fb204100050 con 0x7fb20406ade0 2024-02-20T12:50:48.306 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:48.304+0000 7fb200ff9700 1 -- 172.21.15.64:0/3949255176 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mgrmap(e 9) v1 ==== 37108+0+0 (secure 0 0 0) 0x7fb1f400f070 con 0x7fb20406ade0 2024-02-20T12:50:48.306 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:48.305+0000 7fb200ff9700 1 -- 172.21.15.64:0/3949255176 <== mon.0 v2:172.21.15.64:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) v1 ==== 88+0+0 (secure 0 0 0) 0x7fb1f4010360 con 0x7fb20406ade0 2024-02-20T12:50:48.309 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:48.307+0000 7fb20a90e700 1 -- 172.21.15.64:0/3949255176 >> [v2:172.21.15.64:6800/1362908731,v1:172.21.15.64:6801/1362908731] conn(0x7fb1f0030fa0 msgr2=0x7fb1f0033460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:50:48.309 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:48.307+0000 7fb20a90e700 1 --2- 172.21.15.64:0/3949255176 >> [v2:172.21.15.64:6800/1362908731,v1:172.21.15.64:6801/1362908731] conn(0x7fb1f0030fa0 0x7fb1f0033460 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 rx=0x7fb1f80096a0 tx=0x7fb1f8006b40).stop 2024-02-20T12:50:48.309 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:48.307+0000 7fb20a90e700 1 -- 172.21.15.64:0/3949255176 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb20406ade0 msgr2=0x7fb20410ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:50:48.309 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:48.307+0000 7fb20a90e700 1 --2- 172.21.15.64:0/3949255176 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb20406ade0 0x7fb20410ad70 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 rx=0x7fb1f4011de0 tx=0x7fb1f4006ee0).stop 2024-02-20T12:50:48.309 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:48.307+0000 7fb20a90e700 1 -- 172.21.15.64:0/3949255176 shutdown_connections 2024-02-20T12:50:48.309 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:48.307+0000 7fb20a90e700 1 --2- 172.21.15.64:0/3949255176 >> [v2:172.21.15.64:6800/1362908731,v1:172.21.15.64:6801/1362908731] conn(0x7fb1f0030fa0 0x7fb1f0033460 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:50:48.310 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:48.307+0000 7fb20a90e700 1 --2- 172.21.15.64:0/3949255176 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb20406ade0 0x7fb20410ad70 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:50:48.310 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:48.307+0000 7fb20a90e700 1 -- 172.21.15.64:0/3949255176 >> 172.21.15.64:0/3949255176 conn(0x7fb2040f6680 msgr2=0x7fb2040f8150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:50:48.310 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:48.308+0000 7fb20a90e700 1 -- 172.21.15.64:0/3949255176 shutdown_connections 2024-02-20T12:50:48.310 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:48.308+0000 7fb20a90e700 1 -- 172.21.15.64:0/3949255176 wait complete. 2024-02-20T12:50:48.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:48 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:50:47.867980+0000 mon.smithi064 (mon.0) 69 : audit [INF] from='client.? 172.21.15.64:0/3949255176' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-02-20T12:50:49.572 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:49 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:50:48.306591+0000 mon.smithi064 (mon.0) 70 : audit [INF] from='client.? 172.21.15.64:0/3949255176' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-02-20T12:50:49.573 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:49 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:50:48.306638+0000 mon.smithi064 (mon.0) 71 : cluster [DBG] mgrmap e9: smithi064.vriyor(active, since 25s) 2024-02-20T12:50:49.805 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:49.794+0000 7f6769e75700 1 Processor -- start 2024-02-20T12:50:49.805 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:49.794+0000 7f6769e75700 1 -- start start 2024-02-20T12:50:49.805 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:49.794+0000 7f6769e75700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f67640275f0 0x7f6764101060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-02-20T12:50:49.805 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:49.794+0000 7f6769e75700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67640fca50 con 0x7f67640275f0 2024-02-20T12:50:49.805 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:49.794+0000 7f67637fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f67640275f0 0x7f6764101060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:50:49.805 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:49.794+0000 7f67637fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f67640275f0 0x7f6764101060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42766/0 (socket says 172.21.15.64:42766) 2024-02-20T12:50:49.805 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:49.794+0000 7f67637fe700 1 -- 172.21.15.64:0/3303595770 learned_addr learned my addr 172.21.15.64:0/3303595770 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-20T12:50:49.805 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:49.795+0000 7f67637fe700 1 -- 172.21.15.64:0/3303595770 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6764101550 con 0x7f67640275f0 2024-02-20T12:50:49.805 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:49.795+0000 7f67637fe700 1 --2- 172.21.15.64:0/3303595770 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f67640275f0 0x7f6764101060 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 rx=0x7f675400b5f0 tx=0x7f6754006ee0).ready entity=mon.0 client_cookie=ad1ca7dfb432a3dd server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:50:49.805 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:49.795+0000 7f67627fc700 1 -- 172.21.15.64:0/3303595770 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f675400d600 con 0x7f67640275f0 2024-02-20T12:50:49.805 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:49.795+0000 7f67627fc700 1 -- 172.21.15.64:0/3303595770 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(25 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f675400dbf0 con 0x7f67640275f0 2024-02-20T12:50:49.805 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:49.795+0000 7f67627fc700 1 -- 172.21.15.64:0/3303595770 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6754011bf0 con 0x7f67640275f0 2024-02-20T12:50:49.805 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:49.796+0000 7f6769e75700 1 -- 172.21.15.64:0/3303595770 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f67640275f0 msgr2=0x7f6764101060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:50:49.806 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:49.796+0000 7f6769e75700 1 --2- 172.21.15.64:0/3303595770 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f67640275f0 0x7f6764101060 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 rx=0x7f675400b5f0 tx=0x7f6754006ee0).stop 2024-02-20T12:50:49.806 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:49.796+0000 7f6769e75700 1 -- 172.21.15.64:0/3303595770 shutdown_connections 2024-02-20T12:50:49.806 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:49.796+0000 7f6769e75700 1 --2- 172.21.15.64:0/3303595770 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f67640275f0 0x7f6764101060 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:50:49.806 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:49.796+0000 7f6769e75700 1 -- 172.21.15.64:0/3303595770 >> 172.21.15.64:0/3303595770 conn(0x7f67640f6680 msgr2=0x7f67640f8aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:50:49.806 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:49.796+0000 7f6769e75700 1 -- 172.21.15.64:0/3303595770 shutdown_connections 2024-02-20T12:50:49.806 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:49.796+0000 7f6769e75700 1 -- 172.21.15.64:0/3303595770 wait complete. 2024-02-20T12:50:49.806 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:49.796+0000 7f6769e75700 1 Processor -- start 2024-02-20T12:50:49.806 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:49.797+0000 7f6769e75700 1 -- start start 2024-02-20T12:50:49.806 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:49.797+0000 7f6769e75700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f67640275f0 0x7f676410cf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:50:49.806 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:49.797+0000 7f6769e75700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f676410d420 con 0x7f67640275f0 2024-02-20T12:50:49.806 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:49.797+0000 7f67637fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f67640275f0 0x7f676410cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:50:49.806 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:49.797+0000 7f67637fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f67640275f0 0x7f676410cf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42770/0 (socket says 172.21.15.64:42770) 2024-02-20T12:50:49.806 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:49.797+0000 7f67637fe700 1 -- 172.21.15.64:0/2417497556 learned_addr learned my addr 172.21.15.64:0/2417497556 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-20T12:50:49.806 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:49.797+0000 7f67637fe700 1 -- 172.21.15.64:0/2417497556 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f675400b200 con 0x7f67640275f0 2024-02-20T12:50:49.806 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:49.797+0000 7f67637fe700 1 --2- 172.21.15.64:0/2417497556 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f67640275f0 0x7f676410cf30 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 rx=0x7f6754006800 tx=0x7f6754006ee0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:50:49.806 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:49.798+0000 7f674bfff700 1 -- 172.21.15.64:0/2417497556 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f675400dec0 con 0x7f67640275f0 2024-02-20T12:50:49.807 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:49.798+0000 7f674bfff700 1 -- 172.21.15.64:0/2417497556 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(25 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f675401b8a0 con 0x7f67640275f0 2024-02-20T12:50:49.807 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:49.798+0000 7f6769e75700 1 -- 172.21.15.64:0/2417497556 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f676410d620 con 0x7f67640275f0 2024-02-20T12:50:49.807 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:49.798+0000 7f674bfff700 1 -- 172.21.15.64:0/2417497556 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f675401a690 con 0x7f67640275f0 2024-02-20T12:50:49.807 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:49.798+0000 7f6769e75700 1 -- 172.21.15.64:0/2417497556 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f676410dab0 con 0x7f67640275f0 2024-02-20T12:50:49.807 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:49.798+0000 7f674bfff700 1 -- 172.21.15.64:0/2417497556 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 9) v1 ==== 37108+0+0 (secure 0 0 0) 0x7f675402b070 con 0x7f67640275f0 2024-02-20T12:50:49.807 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:49.799+0000 7f674bfff700 1 --2- 172.21.15.64:0/2417497556 >> [v2:172.21.15.64:6800/1362908731,v1:172.21.15.64:6801/1362908731] conn(0x7f6750030ff0 0x7f67500334b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:50:49.807 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:49.799+0000 7f674bfff700 1 -- 172.21.15.64:0/2417497556 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 893+0+0 (secure 0 0 0) 0x7f6754043b30 con 0x7f67640275f0 2024-02-20T12:50:49.807 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:49.799+0000 7f6762ffd700 1 -- 172.21.15.64:0/2417497556 >> [v2:172.21.15.64:6800/1362908731,v1:172.21.15.64:6801/1362908731] conn(0x7f6750030ff0 msgr2=0x7f67500334b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.64:6800/1362908731 2024-02-20T12:50:49.807 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:49.799+0000 7f6762ffd700 1 --2- 172.21.15.64:0/2417497556 >> [v2:172.21.15.64:6800/1362908731,v1:172.21.15.64:6801/1362908731] conn(0x7f6750030ff0 0x7f67500334b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2024-02-20T12:50:49.807 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:49.799+0000 7f6769e75700 1 -- 172.21.15.64:0/2417497556 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6764045bb0 con 0x7f67640275f0 2024-02-20T12:50:49.807 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:49.803+0000 7f674bfff700 1 -- 172.21.15.64:0/2417497556 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+153237 (secure 0 0 0) 0x7f6754016020 con 0x7f67640275f0 2024-02-20T12:50:50.001 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:50.000+0000 7f6762ffd700 1 -- 172.21.15.64:0/2417497556 >> [v2:172.21.15.64:6800/1362908731,v1:172.21.15.64:6801/1362908731] conn(0x7f6750030ff0 msgr2=0x7f67500334b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.64:6800/1362908731 2024-02-20T12:50:50.001 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:50.000+0000 7f6762ffd700 1 --2- 172.21.15.64:0/2417497556 >> [v2:172.21.15.64:6800/1362908731,v1:172.21.15.64:6801/1362908731] conn(0x7f6750030ff0 0x7f67500334b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2024-02-20T12:50:50.008 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:50.007+0000 7f6769e75700 1 -- 172.21.15.64:0/2417497556 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7f6764106e90 con 0x7f67640275f0 2024-02-20T12:50:50.008 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:50.007+0000 7f674bfff700 1 -- 172.21.15.64:0/2417497556 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) v1 ==== 56+0+103 (secure 0 0 0) 0x7f675401f070 con 0x7f67640275f0 2024-02-20T12:50:50.008 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: { 2024-02-20T12:50:50.008 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: "epoch": 9, 2024-02-20T12:50:50.009 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: "available": true, 2024-02-20T12:50:50.009 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: "active_name": "smithi064.vriyor", 2024-02-20T12:50:50.009 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: "num_standby": 0 2024-02-20T12:50:50.009 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: } 2024-02-20T12:50:50.011 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:50.009+0000 7f6769e75700 1 -- 172.21.15.64:0/2417497556 >> [v2:172.21.15.64:6800/1362908731,v1:172.21.15.64:6801/1362908731] conn(0x7f6750030ff0 msgr2=0x7f67500334b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-20T12:50:50.011 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:50.009+0000 7f6769e75700 1 --2- 172.21.15.64:0/2417497556 >> [v2:172.21.15.64:6800/1362908731,v1:172.21.15.64:6801/1362908731] conn(0x7f6750030ff0 0x7f67500334b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-02-20T12:50:50.011 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:50.009+0000 7f6769e75700 1 -- 172.21.15.64:0/2417497556 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f67640275f0 msgr2=0x7f676410cf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:50:50.011 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:50.009+0000 7f6769e75700 1 --2- 172.21.15.64:0/2417497556 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f67640275f0 0x7f676410cf30 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 rx=0x7f6754006800 tx=0x7f6754006ee0).stop 2024-02-20T12:50:50.011 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:50.009+0000 7f6769e75700 1 -- 172.21.15.64:0/2417497556 shutdown_connections 2024-02-20T12:50:50.011 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:50.009+0000 7f6769e75700 1 --2- 172.21.15.64:0/2417497556 >> [v2:172.21.15.64:6800/1362908731,v1:172.21.15.64:6801/1362908731] conn(0x7f6750030ff0 0x7f67500334b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-02-20T12:50:50.011 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:50.009+0000 7f6769e75700 1 --2- 172.21.15.64:0/2417497556 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f67640275f0 0x7f676410cf30 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:50:50.011 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:50.009+0000 7f6769e75700 1 -- 172.21.15.64:0/2417497556 >> 172.21.15.64:0/2417497556 conn(0x7f67640f6680 msgr2=0x7f67640f8120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:50:50.011 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:50.009+0000 7f6769e75700 1 -- 172.21.15.64:0/2417497556 shutdown_connections 2024-02-20T12:50:50.011 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:50.009+0000 7f6769e75700 1 -- 172.21.15.64:0/2417497556 wait complete. 2024-02-20T12:50:50.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:50 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:50:50.008304+0000 mon.smithi064 (mon.0) 72 : audit [DBG] from='client.? 172.21.15.64:0/2417497556' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-02-20T12:50:50.750 INFO:teuthology.orchestra.run.smithi064.stderr:Waiting for the mgr to restart... 2024-02-20T12:50:50.750 INFO:teuthology.orchestra.run.smithi064.stderr:Waiting for mgr epoch 9... 2024-02-20T12:50:51.455 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:51.432+0000 7f1f3e8dc700 1 Processor -- start 2024-02-20T12:50:51.455 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:51.433+0000 7f1f3e8dc700 1 -- start start 2024-02-20T12:50:51.455 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:51.433+0000 7f1f3e8dc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1f3806ad60 0x7f1f38100eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-02-20T12:50:51.455 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:51.433+0000 7f1f3e8dc700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f380fc8a0 con 0x7f1f3806ad60 2024-02-20T12:50:51.455 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:51.433+0000 7f1f3d8da700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1f3806ad60 0x7f1f38100eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:50:51.455 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:51.433+0000 7f1f3d8da700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1f3806ad60 0x7f1f38100eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42786/0 (socket says 172.21.15.64:42786) 2024-02-20T12:50:51.455 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:51.433+0000 7f1f3d8da700 1 -- 172.21.15.64:0/4052643271 learned_addr learned my addr 172.21.15.64:0/4052643271 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-20T12:50:51.455 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:51.433+0000 7f1f3d8da700 1 -- 172.21.15.64:0/4052643271 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f381013a0 con 0x7f1f3806ad60 2024-02-20T12:50:51.455 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:51.433+0000 7f1f3d8da700 1 --2- 172.21.15.64:0/4052643271 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1f3806ad60 0x7f1f38100eb0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 rx=0x7f1f3400b5f0 tx=0x7f1f34006ee0).ready entity=mon.0 client_cookie=79d47144ed60d228 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:50:51.455 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:51.434+0000 7f1f3c8d8700 1 -- 172.21.15.64:0/4052643271 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1f3400d600 con 0x7f1f3806ad60 2024-02-20T12:50:51.455 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:51.434+0000 7f1f3c8d8700 1 -- 172.21.15.64:0/4052643271 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(25 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f1f3400dbf0 con 0x7f1f3806ad60 2024-02-20T12:50:51.455 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:51.434+0000 7f1f3c8d8700 1 -- 172.21.15.64:0/4052643271 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1f34011bf0 con 0x7f1f3806ad60 2024-02-20T12:50:51.456 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:51.434+0000 7f1f3e8dc700 1 -- 172.21.15.64:0/4052643271 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1f3806ad60 msgr2=0x7f1f38100eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:50:51.456 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:51.434+0000 7f1f3e8dc700 1 --2- 172.21.15.64:0/4052643271 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1f3806ad60 0x7f1f38100eb0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 rx=0x7f1f3400b5f0 tx=0x7f1f34006ee0).stop 2024-02-20T12:50:51.456 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:51.434+0000 7f1f3e8dc700 1 -- 172.21.15.64:0/4052643271 shutdown_connections 2024-02-20T12:50:51.456 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:51.434+0000 7f1f3e8dc700 1 --2- 172.21.15.64:0/4052643271 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1f3806ad60 0x7f1f38100eb0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:50:51.456 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:51.434+0000 7f1f3e8dc700 1 -- 172.21.15.64:0/4052643271 >> 172.21.15.64:0/4052643271 conn(0x7f1f380f64d0 msgr2=0x7f1f380f88f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:50:51.456 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:51.434+0000 7f1f3e8dc700 1 -- 172.21.15.64:0/4052643271 shutdown_connections 2024-02-20T12:50:51.456 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:51.434+0000 7f1f3e8dc700 1 -- 172.21.15.64:0/4052643271 wait complete. 2024-02-20T12:50:51.456 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:51.435+0000 7f1f3e8dc700 1 Processor -- start 2024-02-20T12:50:51.456 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:51.435+0000 7f1f3e8dc700 1 -- start start 2024-02-20T12:50:51.456 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:51.435+0000 7f1f3e8dc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1f3806ad60 0x7f1f3810ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:50:51.456 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:51.435+0000 7f1f3e8dc700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f3810d2f0 con 0x7f1f3806ad60 2024-02-20T12:50:51.456 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:51.435+0000 7f1f3d8da700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1f3806ad60 0x7f1f3810ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:50:51.456 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:51.435+0000 7f1f3d8da700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1f3806ad60 0x7f1f3810ce00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42792/0 (socket says 172.21.15.64:42792) 2024-02-20T12:50:51.457 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:51.435+0000 7f1f3d8da700 1 -- 172.21.15.64:0/3227811962 learned_addr learned my addr 172.21.15.64:0/3227811962 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-20T12:50:51.457 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:51.435+0000 7f1f3d8da700 1 -- 172.21.15.64:0/3227811962 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f3400b200 con 0x7f1f3806ad60 2024-02-20T12:50:51.457 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:51.436+0000 7f1f3d8da700 1 --2- 172.21.15.64:0/3227811962 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1f3806ad60 0x7f1f3810ce00 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 rx=0x7f1f34006800 tx=0x7f1f34006ee0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:50:51.457 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:51.436+0000 7f1f267fc700 1 -- 172.21.15.64:0/3227811962 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1f3400dec0 con 0x7f1f3806ad60 2024-02-20T12:50:51.457 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:51.436+0000 7f1f267fc700 1 -- 172.21.15.64:0/3227811962 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(25 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f1f3401b8a0 con 0x7f1f3806ad60 2024-02-20T12:50:51.457 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:51.436+0000 7f1f3e8dc700 1 -- 172.21.15.64:0/3227811962 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f3810d4f0 con 0x7f1f3806ad60 2024-02-20T12:50:51.457 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:51.436+0000 7f1f267fc700 1 -- 172.21.15.64:0/3227811962 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1f3401a690 con 0x7f1f3806ad60 2024-02-20T12:50:51.457 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:51.436+0000 7f1f3e8dc700 1 -- 172.21.15.64:0/3227811962 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f3810e100 con 0x7f1f3806ad60 2024-02-20T12:50:51.457 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:51.437+0000 7f1f267fc700 1 -- 172.21.15.64:0/3227811962 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 9) v1 ==== 37108+0+0 (secure 0 0 0) 0x7f1f3402b070 con 0x7f1f3806ad60 2024-02-20T12:50:51.457 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:51.437+0000 7f1f267fc700 1 --2- 172.21.15.64:0/3227811962 >> [v2:172.21.15.64:6800/1362908731,v1:172.21.15.64:6801/1362908731] conn(0x7f1f20031040 0x7f1f20033500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:50:51.457 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:51.437+0000 7f1f267fc700 1 -- 172.21.15.64:0/3227811962 --> [v2:172.21.15.64:6800/1362908731,v1:172.21.15.64:6801/1362908731] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1f20033bc0 con 0x7f1f20031040 2024-02-20T12:50:51.457 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:51.437+0000 7f1f267fc700 1 -- 172.21.15.64:0/3227811962 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 893+0+0 (secure 0 0 0) 0x7f1f34042c70 con 0x7f1f3806ad60 2024-02-20T12:50:51.457 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:51.437+0000 7f1f3d0d9700 1 -- 172.21.15.64:0/3227811962 >> [v2:172.21.15.64:6800/1362908731,v1:172.21.15.64:6801/1362908731] conn(0x7f1f20031040 msgr2=0x7f1f20033500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.64:6800/1362908731 2024-02-20T12:50:51.457 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:51.437+0000 7f1f3d0d9700 1 --2- 172.21.15.64:0/3227811962 >> [v2:172.21.15.64:6800/1362908731,v1:172.21.15.64:6801/1362908731] conn(0x7f1f20031040 0x7f1f20033500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2024-02-20T12:50:51.639 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:51.638+0000 7f1f3d0d9700 1 -- 172.21.15.64:0/3227811962 >> [v2:172.21.15.64:6800/1362908731,v1:172.21.15.64:6801/1362908731] conn(0x7f1f20031040 msgr2=0x7f1f20033500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.64:6800/1362908731 2024-02-20T12:50:51.639 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:51.638+0000 7f1f3d0d9700 1 --2- 172.21.15.64:0/3227811962 >> [v2:172.21.15.64:6800/1362908731,v1:172.21.15.64:6801/1362908731] conn(0x7f1f20031040 0x7f1f20033500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2024-02-20T12:50:52.040 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:52.039+0000 7f1f3d0d9700 1 -- 172.21.15.64:0/3227811962 >> [v2:172.21.15.64:6800/1362908731,v1:172.21.15.64:6801/1362908731] conn(0x7f1f20031040 msgr2=0x7f1f20033500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.64:6800/1362908731 2024-02-20T12:50:52.040 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:52.039+0000 7f1f3d0d9700 1 --2- 172.21.15.64:0/3227811962 >> [v2:172.21.15.64:6800/1362908731,v1:172.21.15.64:6801/1362908731] conn(0x7f1f20031040 0x7f1f20033500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.800000 2024-02-20T12:50:52.841 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:52.840+0000 7f1f3d0d9700 1 -- 172.21.15.64:0/3227811962 >> [v2:172.21.15.64:6800/1362908731,v1:172.21.15.64:6801/1362908731] conn(0x7f1f20031040 msgr2=0x7f1f20033500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.64:6800/1362908731 2024-02-20T12:50:52.841 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:52.840+0000 7f1f3d0d9700 1 --2- 172.21.15.64:0/3227811962 >> [v2:172.21.15.64:6800/1362908731,v1:172.21.15.64:6801/1362908731] conn(0x7f1f20031040 0x7f1f20033500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 1.600000 2024-02-20T12:50:53.699 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:53.697+0000 7f1f267fc700 1 -- 172.21.15.64:0/3227811962 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mgrmap(e 10) v1 ==== 36958+0+0 (secure 0 0 0) 0x7f1f3400f070 con 0x7f1f3806ad60 2024-02-20T12:50:53.699 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:53.698+0000 7f1f267fc700 1 -- 172.21.15.64:0/3227811962 >> [v2:172.21.15.64:6800/1362908731,v1:172.21.15.64:6801/1362908731] conn(0x7f1f20031040 msgr2=0x7f1f20033500 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-20T12:50:53.699 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:53.698+0000 7f1f267fc700 1 --2- 172.21.15.64:0/3227811962 >> [v2:172.21.15.64:6800/1362908731,v1:172.21.15.64:6801/1362908731] conn(0x7f1f20031040 0x7f1f20033500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-02-20T12:50:54.701 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:54.699+0000 7f1f267fc700 1 -- 172.21.15.64:0/3227811962 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mgrmap(e 11) v1 ==== 37028+0+0 (secure 0 0 0) 0x7f1f3401b5a0 con 0x7f1f3806ad60 2024-02-20T12:50:54.702 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:54.699+0000 7f1f267fc700 1 --2- 172.21.15.64:0/3227811962 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f1f20034b90 0x7f1f20036f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:50:54.702 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:54.699+0000 7f1f267fc700 1 -- 172.21.15.64:0/3227811962 --> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1f20033bc0 con 0x7f1f20034b90 2024-02-20T12:50:54.702 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:54.699+0000 7f1f3d0d9700 1 --2- 172.21.15.64:0/3227811962 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f1f20034b90 0x7f1f20036f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:50:54.702 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:54.700+0000 7f1f3d0d9700 1 --2- 172.21.15.64:0/3227811962 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f1f20034b90 0x7f1f20036f80 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 rx=0x7f1f280036d0 tx=0x7f1f28006d60).ready entity=mgr.14156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:50:54.702 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:54.700+0000 7f1f267fc700 1 -- 172.21.15.64:0/3227811962 <== mgr.14156 v2:172.21.15.64:6800/1408391390 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+6438 (secure 0 0 0) 0x7f1f20033bc0 con 0x7f1f20034b90 2024-02-20T12:50:54.707 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:54.706+0000 7f1f3e8dc700 1 -- 172.21.15.64:0/3227811962 --> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7f1f3810d980 con 0x7f1f20034b90 2024-02-20T12:50:54.707 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:54.707+0000 7f1f267fc700 1 -- 172.21.15.64:0/3227811962 <== mgr.14156 v2:172.21.15.64:6800/1408391390 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+52 (secure 0 0 0) 0x7f1f3810d980 con 0x7f1f20034b90 2024-02-20T12:50:54.709 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: { 2024-02-20T12:50:54.709 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: "mgrmap_epoch": 11, 2024-02-20T12:50:54.709 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: "initialized": true 2024-02-20T12:50:54.710 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: } 2024-02-20T12:50:54.710 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:54.707+0000 7f1f3e8dc700 1 -- 172.21.15.64:0/3227811962 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f1f20034b90 msgr2=0x7f1f20036f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:50:54.710 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:54.707+0000 7f1f3e8dc700 1 --2- 172.21.15.64:0/3227811962 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f1f20034b90 0x7f1f20036f80 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 rx=0x7f1f280036d0 tx=0x7f1f28006d60).stop 2024-02-20T12:50:54.710 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:54.707+0000 7f1f3e8dc700 1 -- 172.21.15.64:0/3227811962 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1f3806ad60 msgr2=0x7f1f3810ce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:50:54.710 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:54.707+0000 7f1f3e8dc700 1 --2- 172.21.15.64:0/3227811962 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1f3806ad60 0x7f1f3810ce00 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 rx=0x7f1f34006800 tx=0x7f1f34006ee0).stop 2024-02-20T12:50:54.710 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:54.707+0000 7f1f3e8dc700 1 -- 172.21.15.64:0/3227811962 shutdown_connections 2024-02-20T12:50:54.710 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:54.707+0000 7f1f3e8dc700 1 --2- 172.21.15.64:0/3227811962 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f1f20034b90 0x7f1f20036f80 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:50:54.710 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:54.707+0000 7f1f3e8dc700 1 --2- 172.21.15.64:0/3227811962 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1f3806ad60 0x7f1f3810ce00 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:50:54.710 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:54.707+0000 7f1f3e8dc700 1 --2- 172.21.15.64:0/3227811962 >> [v2:172.21.15.64:6800/1362908731,v1:172.21.15.64:6801/1362908731] conn(0x7f1f20031040 0x7f1f20033500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-02-20T12:50:54.710 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:54.707+0000 7f1f3e8dc700 1 -- 172.21.15.64:0/3227811962 >> 172.21.15.64:0/3227811962 conn(0x7f1f380f64d0 msgr2=0x7f1f380f7fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:50:54.710 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:54.707+0000 7f1f3e8dc700 1 -- 172.21.15.64:0/3227811962 shutdown_connections 2024-02-20T12:50:54.711 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:54.707+0000 7f1f3e8dc700 1 -- 172.21.15.64:0/3227811962 wait complete. 2024-02-20T12:50:54.910 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:54 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:50:53.646843+0000 mon.smithi064 (mon.0) 73 : cluster [INF] Active manager daemon smithi064.vriyor restarted 2024-02-20T12:50:54.911 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:54 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:50:53.647277+0000 mon.smithi064 (mon.0) 74 : cluster [INF] Activating manager daemon smithi064.vriyor 2024-02-20T12:50:54.911 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:54 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:50:53.648833+0000 mon.smithi064 (mon.0) 75 : cluster [DBG] osdmap e3: 0 total, 0 up, 0 in 2024-02-20T12:50:54.911 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:54 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:50:53.699711+0000 mon.smithi064 (mon.0) 76 : cluster [DBG] mgrmap e10: smithi064.vriyor(active, starting, since 0.0525462s) 2024-02-20T12:50:54.911 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:54 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:50:53.700628+0000 mon.smithi064 (mon.0) 77 : audit [DBG] from='mgr.14156 172.21.15.64:0/3284974785' entity='mgr.smithi064.vriyor' cmd=[{"prefix": "mon metadata", "id": "smithi064"}]: dispatch 2024-02-20T12:50:54.911 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:54 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:50:53.701226+0000 mon.smithi064 (mon.0) 78 : audit [DBG] from='mgr.14156 172.21.15.64:0/3284974785' entity='mgr.smithi064.vriyor' cmd=[{"prefix": "mgr metadata", "who": "smithi064.vriyor", "id": "smithi064.vriyor"}]: dispatch 2024-02-20T12:50:54.911 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:54 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:50:53.701858+0000 mon.smithi064 (mon.0) 79 : audit [DBG] from='mgr.14156 172.21.15.64:0/3284974785' entity='mgr.smithi064.vriyor' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-20T12:50:54.911 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:54 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:50:53.702037+0000 mon.smithi064 (mon.0) 80 : audit [DBG] from='mgr.14156 172.21.15.64:0/3284974785' entity='mgr.smithi064.vriyor' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-20T12:50:54.911 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:54 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:50:53.702149+0000 mon.smithi064 (mon.0) 81 : audit [DBG] from='mgr.14156 172.21.15.64:0/3284974785' entity='mgr.smithi064.vriyor' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-20T12:50:54.911 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:54 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:50:53.715467+0000 mon.smithi064 (mon.0) 82 : cluster [INF] Manager daemon smithi064.vriyor is now available 2024-02-20T12:50:54.911 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:54 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:50:53.820236+0000 mon.smithi064 (mon.0) 83 : audit [DBG] from='mgr.14156 172.21.15.64:0/3284974785' entity='mgr.smithi064.vriyor' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-20T12:50:54.911 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:54 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:50:53.823491+0000 mon.smithi064 (mon.0) 84 : audit [DBG] from='mgr.14156 172.21.15.64:0/3284974785' entity='mgr.smithi064.vriyor' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-20T12:50:54.911 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:54 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:50:53.841384+0000 mon.smithi064 (mon.0) 85 : audit [INF] from='mgr.14156 172.21.15.64:0/3284974785' entity='mgr.smithi064.vriyor' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi064.vriyor/mirror_snapshot_schedule"}]: dispatch 2024-02-20T12:50:54.911 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:54 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:50:53.843294+0000 mon.smithi064 (mon.0) 86 : audit [INF] from='mgr.14156 172.21.15.64:0/3284974785' entity='mgr.smithi064.vriyor' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi064.vriyor/trash_purge_schedule"}]: dispatch 2024-02-20T12:50:54.911 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:54 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:50:54.457143+0000 mon.smithi064 (mon.0) 87 : audit [INF] from='mgr.14156 172.21.15.64:0/3284974785' entity='mgr.smithi064.vriyor' 2024-02-20T12:50:56.062 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:55 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:50:54.701568+0000 mgr.smithi064.vriyor (mgr.14156) 1 : audit [DBG] from='client.14160 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-02-20T12:50:56.062 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:55 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:50:54.702300+0000 mon.smithi064 (mon.0) 88 : cluster [DBG] mgrmap e11: smithi064.vriyor(active, since 1.05514s) 2024-02-20T12:50:56.062 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:55 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:50:54.707930+0000 mgr.smithi064.vriyor (mgr.14156) 2 : audit [DBG] from='client.14160 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-02-20T12:50:56.563 INFO:teuthology.orchestra.run.smithi064.stderr:mgr epoch 9 is available 2024-02-20T12:50:56.563 INFO:teuthology.orchestra.run.smithi064.stderr:Generating a dashboard self-signed certificate... 2024-02-20T12:50:56.846 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:56 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:50:55.704534+0000 mon.smithi064 (mon.0) 89 : cluster [DBG] mgrmap e12: smithi064.vriyor(active, since 2s) 2024-02-20T12:50:57.190 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:57.183+0000 7eff41fee700 1 Processor -- start 2024-02-20T12:50:57.190 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:57.183+0000 7eff41fee700 1 -- start start 2024-02-20T12:50:57.190 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:57.183+0000 7eff41fee700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7eff3c0fbfc0 0x7eff3c0fc3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-02-20T12:50:57.190 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:57.183+0000 7eff41fee700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff3c0fc8d0 con 0x7eff3c0fbfc0 2024-02-20T12:50:57.190 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:57.184+0000 7eff3b7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7eff3c0fbfc0 0x7eff3c0fc3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:50:57.190 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:57.184+0000 7eff3b7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7eff3c0fbfc0 0x7eff3c0fc3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57444/0 (socket says 172.21.15.64:57444) 2024-02-20T12:50:57.190 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:57.184+0000 7eff3b7fe700 1 -- 172.21.15.64:0/3035311412 learned_addr learned my addr 172.21.15.64:0/3035311412 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-20T12:50:57.191 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:57.184+0000 7eff3b7fe700 1 -- 172.21.15.64:0/3035311412 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff3c0fca10 con 0x7eff3c0fbfc0 2024-02-20T12:50:57.191 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:57.184+0000 7eff3b7fe700 1 --2- 172.21.15.64:0/3035311412 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7eff3c0fbfc0 0x7eff3c0fc3e0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 rx=0x7eff2c00b5f0 tx=0x7eff2c006ee0).ready entity=mon.0 client_cookie=2ac1cea588c5e08 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:50:57.191 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:57.184+0000 7eff3a7fc700 1 -- 172.21.15.64:0/3035311412 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7eff2c00d600 con 0x7eff3c0fbfc0 2024-02-20T12:50:57.191 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:57.184+0000 7eff3a7fc700 1 -- 172.21.15.64:0/3035311412 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(25 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7eff2c00dbf0 con 0x7eff3c0fbfc0 2024-02-20T12:50:57.191 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:57.184+0000 7eff3a7fc700 1 -- 172.21.15.64:0/3035311412 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7eff2c011bf0 con 0x7eff3c0fbfc0 2024-02-20T12:50:57.191 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:57.185+0000 7eff41fee700 1 -- 172.21.15.64:0/3035311412 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7eff3c0fbfc0 msgr2=0x7eff3c0fc3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:50:57.191 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:57.185+0000 7eff41fee700 1 --2- 172.21.15.64:0/3035311412 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7eff3c0fbfc0 0x7eff3c0fc3e0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 rx=0x7eff2c00b5f0 tx=0x7eff2c006ee0).stop 2024-02-20T12:50:57.191 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:57.185+0000 7eff41fee700 1 -- 172.21.15.64:0/3035311412 shutdown_connections 2024-02-20T12:50:57.191 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:57.185+0000 7eff41fee700 1 --2- 172.21.15.64:0/3035311412 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7eff3c0fbfc0 0x7eff3c0fc3e0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:50:57.191 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:57.185+0000 7eff41fee700 1 -- 172.21.15.64:0/3035311412 >> 172.21.15.64:0/3035311412 conn(0x7eff3c0f7770 msgr2=0x7eff3c0f9b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:50:57.191 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:57.185+0000 7eff41fee700 1 -- 172.21.15.64:0/3035311412 shutdown_connections 2024-02-20T12:50:57.192 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:57.185+0000 7eff41fee700 1 -- 172.21.15.64:0/3035311412 wait complete. 2024-02-20T12:50:57.192 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:57.186+0000 7eff41fee700 1 Processor -- start 2024-02-20T12:50:57.192 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:57.186+0000 7eff41fee700 1 -- start start 2024-02-20T12:50:57.192 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:57.186+0000 7eff41fee700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7eff3c0fbfc0 0x7eff3c071c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:50:57.192 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:57.186+0000 7eff41fee700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff3c0700e0 con 0x7eff3c0fbfc0 2024-02-20T12:50:57.192 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:57.186+0000 7eff3b7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7eff3c0fbfc0 0x7eff3c071c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:50:57.192 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:57.186+0000 7eff3b7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7eff3c0fbfc0 0x7eff3c071c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57450/0 (socket says 172.21.15.64:57450) 2024-02-20T12:50:57.192 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:57.186+0000 7eff3b7fe700 1 -- 172.21.15.64:0/4113098195 learned_addr learned my addr 172.21.15.64:0/4113098195 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-20T12:50:57.192 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:57.186+0000 7eff3b7fe700 1 -- 172.21.15.64:0/4113098195 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff2c00b200 con 0x7eff3c0fbfc0 2024-02-20T12:50:57.192 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:57.187+0000 7eff3b7fe700 1 --2- 172.21.15.64:0/4113098195 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7eff3c0fbfc0 0x7eff3c071c60 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 rx=0x7eff2c006800 tx=0x7eff2c006ee0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:50:57.193 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:57.187+0000 7eff33fff700 1 -- 172.21.15.64:0/4113098195 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7eff2c00dec0 con 0x7eff3c0fbfc0 2024-02-20T12:50:57.193 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:57.187+0000 7eff33fff700 1 -- 172.21.15.64:0/4113098195 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(25 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7eff2c01b8a0 con 0x7eff3c0fbfc0 2024-02-20T12:50:57.193 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:57.187+0000 7eff41fee700 1 -- 172.21.15.64:0/4113098195 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff3c070280 con 0x7eff3c0fbfc0 2024-02-20T12:50:57.193 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:57.187+0000 7eff33fff700 1 -- 172.21.15.64:0/4113098195 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7eff2c01a7b0 con 0x7eff3c0fbfc0 2024-02-20T12:50:57.193 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:57.187+0000 7eff41fee700 1 -- 172.21.15.64:0/4113098195 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff3c070710 con 0x7eff3c0fbfc0 2024-02-20T12:50:57.193 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:57.188+0000 7eff33fff700 1 -- 172.21.15.64:0/4113098195 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 12) v1 ==== 37108+0+0 (secure 0 0 0) 0x7eff2c02b070 con 0x7eff3c0fbfc0 2024-02-20T12:50:57.193 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:57.188+0000 7eff33fff700 1 --2- 172.21.15.64:0/4113098195 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7eff1c031040 0x7eff1c033500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:50:57.193 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:57.188+0000 7eff33fff700 1 -- 172.21.15.64:0/4113098195 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1065+0+0 (secure 0 0 0) 0x7eff2c042c60 con 0x7eff3c0fbfc0 2024-02-20T12:50:57.193 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:57.188+0000 7eff3affd700 1 --2- 172.21.15.64:0/4113098195 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7eff1c031040 0x7eff1c033500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:50:57.193 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:57.188+0000 7eff41fee700 1 -- 172.21.15.64:0/4113098195 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff3c0591e0 con 0x7eff3c0fbfc0 2024-02-20T12:50:57.193 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:57.188+0000 7eff3affd700 1 --2- 172.21.15.64:0/4113098195 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7eff1c031040 0x7eff1c033500 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 rx=0x7eff240096a0 tx=0x7eff24006b40).ready entity=mgr.14156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:50:57.194 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:57.193+0000 7eff33fff700 1 -- 172.21.15.64:0/4113098195 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+153237 (secure 0 0 0) 0x7eff2c01f070 con 0x7eff3c0fbfc0 2024-02-20T12:50:57.362 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:57.361+0000 7eff41fee700 1 -- 172.21.15.64:0/4113098195 --> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) v1 -- 0x7eff3c104c50 con 0x7eff1c031040 2024-02-20T12:50:57.449 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:57.448+0000 7eff33fff700 1 -- 172.21.15.64:0/4113098195 <== mgr.14156 v2:172.21.15.64:6800/1408391390 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+32 (secure 0 0 0) 0x7eff3c104c50 con 0x7eff1c031040 2024-02-20T12:50:57.449 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: Self-signed certificate created 2024-02-20T12:50:57.452 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:57.449+0000 7eff41fee700 1 -- 172.21.15.64:0/4113098195 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7eff1c031040 msgr2=0x7eff1c033500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:50:57.452 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:57.449+0000 7eff41fee700 1 --2- 172.21.15.64:0/4113098195 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7eff1c031040 0x7eff1c033500 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 rx=0x7eff240096a0 tx=0x7eff24006b40).stop 2024-02-20T12:50:57.452 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:57.450+0000 7eff41fee700 1 -- 172.21.15.64:0/4113098195 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7eff3c0fbfc0 msgr2=0x7eff3c071c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:50:57.452 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:57.450+0000 7eff41fee700 1 --2- 172.21.15.64:0/4113098195 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7eff3c0fbfc0 0x7eff3c071c60 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 rx=0x7eff2c006800 tx=0x7eff2c006ee0).stop 2024-02-20T12:50:57.452 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:57.450+0000 7eff41fee700 1 -- 172.21.15.64:0/4113098195 shutdown_connections 2024-02-20T12:50:57.452 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:57.450+0000 7eff41fee700 1 --2- 172.21.15.64:0/4113098195 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7eff1c031040 0x7eff1c033500 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:50:57.452 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:57.450+0000 7eff41fee700 1 --2- 172.21.15.64:0/4113098195 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7eff3c0fbfc0 0x7eff3c071c60 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:50:57.452 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:57.450+0000 7eff41fee700 1 -- 172.21.15.64:0/4113098195 >> 172.21.15.64:0/4113098195 conn(0x7eff3c0f7770 msgr2=0x7eff3c0f9210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:50:57.452 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:57.450+0000 7eff41fee700 1 -- 172.21.15.64:0/4113098195 shutdown_connections 2024-02-20T12:50:57.452 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:57.450+0000 7eff41fee700 1 -- 172.21.15.64:0/4113098195 wait complete. 2024-02-20T12:50:58.130 INFO:teuthology.orchestra.run.smithi064.stderr:Creating initial admin user... 2024-02-20T12:50:58.703 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:58.682+0000 7f07383c3700 1 Processor -- start 2024-02-20T12:50:58.703 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:58.682+0000 7f07383c3700 1 -- start start 2024-02-20T12:50:58.703 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:58.683+0000 7f07383c3700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0730027670 0x7f07300febe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-02-20T12:50:58.703 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:58.683+0000 7f07383c3700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07300ff0d0 con 0x7f0730027670 2024-02-20T12:50:58.703 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:58.683+0000 7f073615f700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0730027670 0x7f07300febe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:50:58.704 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:58.683+0000 7f073615f700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0730027670 0x7f07300febe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57454/0 (socket says 172.21.15.64:57454) 2024-02-20T12:50:58.704 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:58.683+0000 7f073615f700 1 -- 172.21.15.64:0/703172337 learned_addr learned my addr 172.21.15.64:0/703172337 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-20T12:50:58.704 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:58.683+0000 7f073615f700 1 -- 172.21.15.64:0/703172337 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07300ff210 con 0x7f0730027670 2024-02-20T12:50:58.704 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:58.683+0000 7f073615f700 1 --2- 172.21.15.64:0/703172337 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0730027670 0x7f07300febe0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 rx=0x7f072000b5f0 tx=0x7f0720006ee0).ready entity=mon.0 client_cookie=b904fc46088b8935 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:50:58.704 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:58.683+0000 7f073515d700 1 -- 172.21.15.64:0/703172337 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f072000d600 con 0x7f0730027670 2024-02-20T12:50:58.704 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:58.683+0000 7f073515d700 1 -- 172.21.15.64:0/703172337 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(25 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f072000dbf0 con 0x7f0730027670 2024-02-20T12:50:58.704 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:58.683+0000 7f073515d700 1 -- 172.21.15.64:0/703172337 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0720011bf0 con 0x7f0730027670 2024-02-20T12:50:58.704 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:58.684+0000 7f07383c3700 1 -- 172.21.15.64:0/703172337 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0730027670 msgr2=0x7f07300febe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:50:58.704 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:58.684+0000 7f07383c3700 1 --2- 172.21.15.64:0/703172337 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0730027670 0x7f07300febe0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 rx=0x7f072000b5f0 tx=0x7f0720006ee0).stop 2024-02-20T12:50:58.704 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:58.684+0000 7f07383c3700 1 -- 172.21.15.64:0/703172337 shutdown_connections 2024-02-20T12:50:58.705 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:58.684+0000 7f07383c3700 1 --2- 172.21.15.64:0/703172337 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0730027670 0x7f07300febe0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:50:58.705 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:58.684+0000 7f07383c3700 1 -- 172.21.15.64:0/703172337 >> 172.21.15.64:0/703172337 conn(0x7f07300f6680 msgr2=0x7f07300f8aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:50:58.705 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:58.684+0000 7f07383c3700 1 -- 172.21.15.64:0/703172337 shutdown_connections 2024-02-20T12:50:58.705 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:58.684+0000 7f07383c3700 1 -- 172.21.15.64:0/703172337 wait complete. 2024-02-20T12:50:58.705 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:58.685+0000 7f07383c3700 1 Processor -- start 2024-02-20T12:50:58.705 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:58.685+0000 7f07383c3700 1 -- start start 2024-02-20T12:50:58.705 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:58.685+0000 7f07383c3700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0730027670 0x7f07300fc660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:50:58.705 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:58.685+0000 7f07383c3700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07300faae0 con 0x7f0730027670 2024-02-20T12:50:58.705 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:58.685+0000 7f073615f700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0730027670 0x7f07300fc660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:50:58.705 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:58.685+0000 7f073615f700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0730027670 0x7f07300fc660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57466/0 (socket says 172.21.15.64:57466) 2024-02-20T12:50:58.705 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:58.685+0000 7f073615f700 1 -- 172.21.15.64:0/4178704555 learned_addr learned my addr 172.21.15.64:0/4178704555 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-20T12:50:58.705 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:58.685+0000 7f073615f700 1 -- 172.21.15.64:0/4178704555 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f072000b200 con 0x7f0730027670 2024-02-20T12:50:58.706 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:58.685+0000 7f073615f700 1 --2- 172.21.15.64:0/4178704555 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0730027670 0x7f07300fc660 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 rx=0x7f0720006800 tx=0x7f0720006ee0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:50:58.706 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:58.685+0000 7f071effd700 1 -- 172.21.15.64:0/4178704555 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f072000dec0 con 0x7f0730027670 2024-02-20T12:50:58.706 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:58.685+0000 7f071effd700 1 -- 172.21.15.64:0/4178704555 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(25 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f072001b8a0 con 0x7f0730027670 2024-02-20T12:50:58.706 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:58.685+0000 7f07383c3700 1 -- 172.21.15.64:0/4178704555 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07300fac80 con 0x7f0730027670 2024-02-20T12:50:58.706 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:58.685+0000 7f071effd700 1 -- 172.21.15.64:0/4178704555 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f072001a7b0 con 0x7f0730027670 2024-02-20T12:50:58.706 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:58.686+0000 7f07383c3700 1 -- 172.21.15.64:0/4178704555 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07300fb110 con 0x7f0730027670 2024-02-20T12:50:58.706 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:58.686+0000 7f07383c3700 1 -- 172.21.15.64:0/4178704555 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0730045bb0 con 0x7f0730027670 2024-02-20T12:50:58.706 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:58.687+0000 7f071effd700 1 -- 172.21.15.64:0/4178704555 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 12) v1 ==== 37108+0+0 (secure 0 0 0) 0x7f072002b070 con 0x7f0730027670 2024-02-20T12:50:58.706 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:58.687+0000 7f071effd700 1 --2- 172.21.15.64:0/4178704555 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f0718030ff0 0x7f07180334b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:50:58.706 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:58.687+0000 7f071effd700 1 -- 172.21.15.64:0/4178704555 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1065+0+0 (secure 0 0 0) 0x7f0720043120 con 0x7f0730027670 2024-02-20T12:50:58.706 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:58.687+0000 7f073595e700 1 --2- 172.21.15.64:0/4178704555 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f0718030ff0 0x7f07180334b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:50:58.706 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:58.687+0000 7f073595e700 1 --2- 172.21.15.64:0/4178704555 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f0718030ff0 0x7f07180334b0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0x7f072c0096a0 tx=0x7f072c006b40).ready entity=mgr.14156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:50:58.707 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:58.691+0000 7f071effd700 1 -- 172.21.15.64:0/4178704555 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+153237 (secure 0 0 0) 0x7f072001ba10 con 0x7f0730027670 2024-02-20T12:50:58.812 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:58 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:50:57.362573+0000 mgr.smithi064.vriyor (mgr.14156) 3 : audit [DBG] from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-02-20T12:50:58.812 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:58 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:50:57.447071+0000 mon.smithi064 (mon.0) 90 : audit [INF] from='mgr.14156 172.21.15.64:0/3284974785' entity='mgr.smithi064.vriyor' 2024-02-20T12:50:58.812 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:50:58 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:50:57.449009+0000 mon.smithi064 (mon.0) 91 : audit [INF] from='mgr.14156 172.21.15.64:0/3284974785' entity='mgr.smithi064.vriyor' 2024-02-20T12:50:58.871 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:58.870+0000 7f07383c3700 1 -- 172.21.15.64:0/4178704555 --> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f0730104de0 con 0x7f0718030ff0 2024-02-20T12:50:59.116 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:59.115+0000 7f071effd700 1 -- 172.21.15.64:0/4178704555 <== mgr.14156 v2:172.21.15.64:6800/1408391390 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+252 (secure 0 0 0) 0x7f0730104de0 con 0x7f0718030ff0 2024-02-20T12:50:59.117 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: {"username": "admin", "password": "$2b$12$JqnZMBuGvRDNd2hgkinNeeUsC5bEh3IgLfFjeTy7B9RTXJy161u2i", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1708433459, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-02-20T12:50:59.119 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:59.117+0000 7f07383c3700 1 -- 172.21.15.64:0/4178704555 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f0718030ff0 msgr2=0x7f07180334b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:50:59.119 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:59.117+0000 7f07383c3700 1 --2- 172.21.15.64:0/4178704555 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f0718030ff0 0x7f07180334b0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0x7f072c0096a0 tx=0x7f072c006b40).stop 2024-02-20T12:50:59.119 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:59.117+0000 7f07383c3700 1 -- 172.21.15.64:0/4178704555 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0730027670 msgr2=0x7f07300fc660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:50:59.119 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:59.117+0000 7f07383c3700 1 --2- 172.21.15.64:0/4178704555 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0730027670 0x7f07300fc660 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 rx=0x7f0720006800 tx=0x7f0720006ee0).stop 2024-02-20T12:50:59.119 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:59.117+0000 7f07383c3700 1 -- 172.21.15.64:0/4178704555 shutdown_connections 2024-02-20T12:50:59.119 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:59.117+0000 7f07383c3700 1 --2- 172.21.15.64:0/4178704555 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f0718030ff0 0x7f07180334b0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:50:59.119 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:59.117+0000 7f07383c3700 1 --2- 172.21.15.64:0/4178704555 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0730027670 0x7f07300fc660 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:50:59.119 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:59.117+0000 7f07383c3700 1 -- 172.21.15.64:0/4178704555 >> 172.21.15.64:0/4178704555 conn(0x7f07300f6680 msgr2=0x7f07300f81a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:50:59.119 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:59.118+0000 7f07383c3700 1 -- 172.21.15.64:0/4178704555 shutdown_connections 2024-02-20T12:50:59.119 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:50:59.118+0000 7f07383c3700 1 -- 172.21.15.64:0/4178704555 wait complete. 2024-02-20T12:50:59.896 INFO:teuthology.orchestra.run.smithi064.stderr:Fetching dashboard port number... 2024-02-20T12:51:00.186 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:51:00 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:50:58.871810+0000 mgr.smithi064.vriyor (mgr.14156) 4 : audit [DBG] from='client.14168 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-02-20T12:51:00.186 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:51:00 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:50:59.116402+0000 mon.smithi064 (mon.0) 92 : audit [INF] from='mgr.14156 172.21.15.64:0/3284974785' entity='mgr.smithi064.vriyor' 2024-02-20T12:51:00.557 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:51:00.553+0000 7fed26528700 1 Processor -- start 2024-02-20T12:51:00.557 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:51:00.553+0000 7fed26528700 1 -- start start 2024-02-20T12:51:00.557 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:51:00.553+0000 7fed26528700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fed20071c60 0x7fed20070160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-02-20T12:51:00.558 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:51:00.553+0000 7fed26528700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed20072080 con 0x7fed20071c60 2024-02-20T12:51:00.558 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:51:00.553+0000 7fed1ffff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fed20071c60 0x7fed20070160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:51:00.558 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:51:00.553+0000 7fed1ffff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fed20071c60 0x7fed20070160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57482/0 (socket says 172.21.15.64:57482) 2024-02-20T12:51:00.558 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:51:00.553+0000 7fed1ffff700 1 -- 172.21.15.64:0/1262118609 learned_addr learned my addr 172.21.15.64:0/1262118609 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-20T12:51:00.558 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:51:00.554+0000 7fed1ffff700 1 -- 172.21.15.64:0/1262118609 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed20070650 con 0x7fed20071c60 2024-02-20T12:51:00.558 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:51:00.554+0000 7fed1ffff700 1 --2- 172.21.15.64:0/1262118609 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fed20071c60 0x7fed20070160 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 rx=0x7fed1000b5f0 tx=0x7fed10006ee0).ready entity=mon.0 client_cookie=78be2c1f378ea3b4 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:51:00.558 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:51:00.554+0000 7fed1effd700 1 -- 172.21.15.64:0/1262118609 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fed1000d600 con 0x7fed20071c60 2024-02-20T12:51:00.558 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:51:00.554+0000 7fed1effd700 1 -- 172.21.15.64:0/1262118609 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(25 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7fed1000dbf0 con 0x7fed20071c60 2024-02-20T12:51:00.558 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:51:00.554+0000 7fed1effd700 1 -- 172.21.15.64:0/1262118609 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fed10011bf0 con 0x7fed20071c60 2024-02-20T12:51:00.558 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:51:00.555+0000 7fed26528700 1 -- 172.21.15.64:0/1262118609 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fed20071c60 msgr2=0x7fed20070160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:51:00.558 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:51:00.555+0000 7fed26528700 1 --2- 172.21.15.64:0/1262118609 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fed20071c60 0x7fed20070160 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 rx=0x7fed1000b5f0 tx=0x7fed10006ee0).stop 2024-02-20T12:51:00.558 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:51:00.555+0000 7fed26528700 1 -- 172.21.15.64:0/1262118609 shutdown_connections 2024-02-20T12:51:00.558 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:51:00.555+0000 7fed26528700 1 --2- 172.21.15.64:0/1262118609 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fed20071c60 0x7fed20070160 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:51:00.558 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:51:00.555+0000 7fed26528700 1 -- 172.21.15.64:0/1262118609 >> 172.21.15.64:0/1262118609 conn(0x7fed200f7790 msgr2=0x7fed200f9bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:51:00.558 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:51:00.555+0000 7fed26528700 1 -- 172.21.15.64:0/1262118609 shutdown_connections 2024-02-20T12:51:00.559 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:51:00.555+0000 7fed26528700 1 -- 172.21.15.64:0/1262118609 wait complete. 2024-02-20T12:51:00.559 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:51:00.555+0000 7fed26528700 1 Processor -- start 2024-02-20T12:51:00.559 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:51:00.556+0000 7fed26528700 1 -- start start 2024-02-20T12:51:00.559 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:51:00.556+0000 7fed26528700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fed20071c60 0x7fed2010cf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:51:00.559 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:51:00.556+0000 7fed26528700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed2010d410 con 0x7fed20071c60 2024-02-20T12:51:00.559 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:51:00.556+0000 7fed1ffff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fed20071c60 0x7fed2010cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:51:00.559 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:51:00.556+0000 7fed1ffff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fed20071c60 0x7fed2010cf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57492/0 (socket says 172.21.15.64:57492) 2024-02-20T12:51:00.559 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:51:00.556+0000 7fed1ffff700 1 -- 172.21.15.64:0/4107180405 learned_addr learned my addr 172.21.15.64:0/4107180405 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-20T12:51:00.560 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:51:00.556+0000 7fed1ffff700 1 -- 172.21.15.64:0/4107180405 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed1000b200 con 0x7fed20071c60 2024-02-20T12:51:00.560 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:51:00.556+0000 7fed1ffff700 1 --2- 172.21.15.64:0/4107180405 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fed20071c60 0x7fed2010cf20 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 rx=0x7fed1000bdc0 tx=0x7fed10006ee0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:51:00.560 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:51:00.557+0000 7fed1cff9700 1 -- 172.21.15.64:0/4107180405 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fed1001b410 con 0x7fed20071c60 2024-02-20T12:51:00.560 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:51:00.557+0000 7fed1cff9700 1 -- 172.21.15.64:0/4107180405 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(25 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7fed1001ba00 con 0x7fed20071c60 2024-02-20T12:51:00.560 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:51:00.557+0000 7fed26528700 1 -- 172.21.15.64:0/4107180405 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed2010d610 con 0x7fed20071c60 2024-02-20T12:51:00.560 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:51:00.557+0000 7fed1cff9700 1 -- 172.21.15.64:0/4107180405 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fed1001a6b0 con 0x7fed20071c60 2024-02-20T12:51:00.560 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:51:00.557+0000 7fed26528700 1 -- 172.21.15.64:0/4107180405 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed2010daa0 con 0x7fed20071c60 2024-02-20T12:51:00.560 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:51:00.558+0000 7fed1cff9700 1 -- 172.21.15.64:0/4107180405 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 37151+0+0 (secure 0 0 0) 0x7fed1002b070 con 0x7fed20071c60 2024-02-20T12:51:00.560 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:51:00.558+0000 7fed1cff9700 1 --2- 172.21.15.64:0/4107180405 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fed0c0310e0 0x7fed0c0335a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:51:00.560 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:51:00.558+0000 7fed1cff9700 1 -- 172.21.15.64:0/4107180405 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1065+0+0 (secure 0 0 0) 0x7fed10042cb0 con 0x7fed20071c60 2024-02-20T12:51:00.560 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:51:00.558+0000 7fed26528700 1 -- 172.21.15.64:0/4107180405 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed20045bb0 con 0x7fed20071c60 2024-02-20T12:51:00.560 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:51:00.558+0000 7fed1f7fe700 1 --2- 172.21.15.64:0/4107180405 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fed0c0310e0 0x7fed0c0335a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:51:00.560 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:51:00.558+0000 7fed1f7fe700 1 --2- 172.21.15.64:0/4107180405 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fed0c0310e0 0x7fed0c0335a0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0x7fed140096a0 tx=0x7fed14006b40).ready entity=mgr.14156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:51:00.563 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:51:00.562+0000 7fed1cff9700 1 -- 172.21.15.64:0/4107180405 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+153237 (secure 0 0 0) 0x7fed1001f070 con 0x7fed20071c60 2024-02-20T12:51:00.721 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:51:00.720+0000 7fed26528700 1 -- 172.21.15.64:0/4107180405 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) v1 -- 0x7fed20102140 con 0x7fed20071c60 2024-02-20T12:51:00.721 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:51:00.720+0000 7fed1cff9700 1 -- 172.21.15.64:0/4107180405 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v10) v1 ==== 112+0+5 (secure 0 0 0) 0x7fed100330b0 con 0x7fed20071c60 2024-02-20T12:51:00.722 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 8443 2024-02-20T12:51:00.724 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:51:00.722+0000 7fed26528700 1 -- 172.21.15.64:0/4107180405 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fed0c0310e0 msgr2=0x7fed0c0335a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:51:00.724 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:51:00.722+0000 7fed26528700 1 --2- 172.21.15.64:0/4107180405 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fed0c0310e0 0x7fed0c0335a0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0x7fed140096a0 tx=0x7fed14006b40).stop 2024-02-20T12:51:00.724 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:51:00.722+0000 7fed26528700 1 -- 172.21.15.64:0/4107180405 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fed20071c60 msgr2=0x7fed2010cf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:51:00.724 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:51:00.722+0000 7fed26528700 1 --2- 172.21.15.64:0/4107180405 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fed20071c60 0x7fed2010cf20 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 rx=0x7fed1000bdc0 tx=0x7fed10006ee0).stop 2024-02-20T12:51:00.724 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:51:00.722+0000 7fed26528700 1 -- 172.21.15.64:0/4107180405 shutdown_connections 2024-02-20T12:51:00.724 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:51:00.722+0000 7fed26528700 1 --2- 172.21.15.64:0/4107180405 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fed0c0310e0 0x7fed0c0335a0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:51:00.724 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:51:00.722+0000 7fed26528700 1 --2- 172.21.15.64:0/4107180405 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fed20071c60 0x7fed2010cf20 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:51:00.724 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:51:00.722+0000 7fed26528700 1 -- 172.21.15.64:0/4107180405 >> 172.21.15.64:0/4107180405 conn(0x7fed200f7790 msgr2=0x7fed200f91d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:51:00.724 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:51:00.722+0000 7fed26528700 1 -- 172.21.15.64:0/4107180405 shutdown_connections 2024-02-20T12:51:00.724 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:51:00.722+0000 7fed26528700 1 -- 172.21.15.64:0/4107180405 wait complete. 2024-02-20T12:51:01.506 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:51:01 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:51:00.120030+0000 mon.smithi064 (mon.0) 93 : cluster [DBG] mgrmap e13: smithi064.vriyor(active, since 6s) 2024-02-20T12:51:01.506 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:51:01 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:51:00.721709+0000 mon.smithi064 (mon.0) 94 : audit [DBG] from='client.? 172.21.15.64:0/4107180405' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-02-20T12:51:01.513 INFO:teuthology.orchestra.run.smithi064.stderr:systemctl: disabled 2024-02-20T12:51:01.522 INFO:teuthology.orchestra.run.smithi064.stderr:systemctl: inactive 2024-02-20T12:51:01.522 INFO:teuthology.orchestra.run.smithi064.stderr:firewalld.service is not enabled 2024-02-20T12:51:01.523 INFO:teuthology.orchestra.run.smithi064.stderr:Not possible to open ports <[8443]>. firewalld.service is not available 2024-02-20T12:51:01.526 INFO:teuthology.orchestra.run.smithi064.stderr:Ceph Dashboard is now available at: 2024-02-20T12:51:01.526 INFO:teuthology.orchestra.run.smithi064.stderr: 2024-02-20T12:51:01.526 INFO:teuthology.orchestra.run.smithi064.stderr: URL: https://smithi064.front.sepia.ceph.com:8443/ 2024-02-20T12:51:01.526 INFO:teuthology.orchestra.run.smithi064.stderr: User: admin 2024-02-20T12:51:01.526 INFO:teuthology.orchestra.run.smithi064.stderr: Password: hk0vh1rzwh 2024-02-20T12:51:01.526 INFO:teuthology.orchestra.run.smithi064.stderr: 2024-02-20T12:51:01.526 INFO:teuthology.orchestra.run.smithi064.stderr:Enabling autotune for osd_memory_target 2024-02-20T12:51:02.149 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:51:02.124+0000 7fdcde748700 1 Processor -- start 2024-02-20T12:51:02.149 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:51:02.124+0000 7fdcde748700 1 -- start start 2024-02-20T12:51:02.149 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:51:02.124+0000 7fdcde748700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdcd806ad60 0x7fdcd8101060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-02-20T12:51:02.149 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:51:02.125+0000 7fdcde748700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcd80fca50 con 0x7fdcd806ad60 2024-02-20T12:51:02.149 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:51:02.125+0000 7fdcd7fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdcd806ad60 0x7fdcd8101060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:51:02.149 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:51:02.125+0000 7fdcd7fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdcd806ad60 0x7fdcd8101060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:46850/0 (socket says 172.21.15.64:46850) 2024-02-20T12:51:02.149 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:51:02.125+0000 7fdcd7fff700 1 -- 172.21.15.64:0/2145537967 learned_addr learned my addr 172.21.15.64:0/2145537967 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-20T12:51:02.149 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:51:02.125+0000 7fdcd7fff700 1 -- 172.21.15.64:0/2145537967 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcd8101550 con 0x7fdcd806ad60 2024-02-20T12:51:02.150 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:51:02.125+0000 7fdcd7fff700 1 --2- 172.21.15.64:0/2145537967 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdcd806ad60 0x7fdcd8101060 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 rx=0x7fdcc800b5f0 tx=0x7fdcc8006ee0).ready entity=mon.0 client_cookie=a078e474bfdc58f8 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:51:02.150 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:51:02.125+0000 7fdcd6ffd700 1 -- 172.21.15.64:0/2145537967 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fdcc800d600 con 0x7fdcd806ad60 2024-02-20T12:51:02.150 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:51:02.125+0000 7fdcd6ffd700 1 -- 172.21.15.64:0/2145537967 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(25 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7fdcc800dbf0 con 0x7fdcd806ad60 2024-02-20T12:51:02.150 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:51:02.125+0000 7fdcd6ffd700 1 -- 172.21.15.64:0/2145537967 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fdcc8011bf0 con 0x7fdcd806ad60 2024-02-20T12:51:02.150 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:51:02.126+0000 7fdcde748700 1 -- 172.21.15.64:0/2145537967 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdcd806ad60 msgr2=0x7fdcd8101060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:51:02.150 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:51:02.126+0000 7fdcde748700 1 --2- 172.21.15.64:0/2145537967 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdcd806ad60 0x7fdcd8101060 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 rx=0x7fdcc800b5f0 tx=0x7fdcc8006ee0).stop 2024-02-20T12:51:02.150 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:51:02.126+0000 7fdcde748700 1 -- 172.21.15.64:0/2145537967 shutdown_connections 2024-02-20T12:51:02.150 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:51:02.126+0000 7fdcde748700 1 --2- 172.21.15.64:0/2145537967 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdcd806ad60 0x7fdcd8101060 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:51:02.150 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:51:02.126+0000 7fdcde748700 1 -- 172.21.15.64:0/2145537967 >> 172.21.15.64:0/2145537967 conn(0x7fdcd80f6680 msgr2=0x7fdcd80f8aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:51:02.150 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:51:02.126+0000 7fdcde748700 1 -- 172.21.15.64:0/2145537967 shutdown_connections 2024-02-20T12:51:02.150 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:51:02.126+0000 7fdcde748700 1 -- 172.21.15.64:0/2145537967 wait complete. 2024-02-20T12:51:02.150 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:51:02.126+0000 7fdcde748700 1 Processor -- start 2024-02-20T12:51:02.150 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:51:02.126+0000 7fdcde748700 1 -- start start 2024-02-20T12:51:02.150 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:51:02.127+0000 7fdcde748700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdcd806ad60 0x7fdcd810cf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:51:02.150 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:51:02.127+0000 7fdcde748700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcd810d420 con 0x7fdcd806ad60 2024-02-20T12:51:02.151 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:51:02.127+0000 7fdcd7fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdcd806ad60 0x7fdcd810cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:51:02.151 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:51:02.127+0000 7fdcd7fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdcd806ad60 0x7fdcd810cf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:46852/0 (socket says 172.21.15.64:46852) 2024-02-20T12:51:02.151 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:51:02.127+0000 7fdcd7fff700 1 -- 172.21.15.64:0/1175692139 learned_addr learned my addr 172.21.15.64:0/1175692139 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-20T12:51:02.151 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:51:02.127+0000 7fdcd7fff700 1 -- 172.21.15.64:0/1175692139 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcc800b200 con 0x7fdcd806ad60 2024-02-20T12:51:02.151 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:51:02.127+0000 7fdcd7fff700 1 --2- 172.21.15.64:0/1175692139 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdcd806ad60 0x7fdcd810cf30 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 rx=0x7fdcc8006800 tx=0x7fdcc8006ee0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:51:02.151 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:51:02.127+0000 7fdcd4ff9700 1 -- 172.21.15.64:0/1175692139 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fdcc800dec0 con 0x7fdcd806ad60 2024-02-20T12:51:02.151 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:51:02.127+0000 7fdcd4ff9700 1 -- 172.21.15.64:0/1175692139 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(25 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7fdcc801b8a0 con 0x7fdcd806ad60 2024-02-20T12:51:02.151 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:51:02.127+0000 7fdcde748700 1 -- 172.21.15.64:0/1175692139 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdcd810d620 con 0x7fdcd806ad60 2024-02-20T12:51:02.151 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:51:02.127+0000 7fdcd4ff9700 1 -- 172.21.15.64:0/1175692139 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fdcc801a690 con 0x7fdcd806ad60 2024-02-20T12:51:02.151 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:51:02.127+0000 7fdcde748700 1 -- 172.21.15.64:0/1175692139 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdcd810dab0 con 0x7fdcd806ad60 2024-02-20T12:51:02.151 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:51:02.128+0000 7fdcd4ff9700 1 -- 172.21.15.64:0/1175692139 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 37151+0+0 (secure 0 0 0) 0x7fdcc802b070 con 0x7fdcd806ad60 2024-02-20T12:51:02.151 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:51:02.128+0000 7fdcd4ff9700 1 --2- 172.21.15.64:0/1175692139 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fdcc4031090 0x7fdcc4033550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:51:02.151 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:51:02.128+0000 7fdcd4ff9700 1 -- 172.21.15.64:0/1175692139 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1065+0+0 (secure 0 0 0) 0x7fdcc8042d30 con 0x7fdcd806ad60 2024-02-20T12:51:02.151 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:51:02.128+0000 7fdcde748700 1 -- 172.21.15.64:0/1175692139 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdcd8045bb0 con 0x7fdcd806ad60 2024-02-20T12:51:02.151 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:51:02.128+0000 7fdcd77fe700 1 --2- 172.21.15.64:0/1175692139 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fdcc4031090 0x7fdcc4033550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:51:02.152 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:51:02.129+0000 7fdcd77fe700 1 --2- 172.21.15.64:0/1175692139 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fdcc4031090 0x7fdcc4033550 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 rx=0x7fdccc0096a0 tx=0x7fdccc006b40).ready entity=mgr.14156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:51:02.152 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:51:02.133+0000 7fdcd4ff9700 1 -- 172.21.15.64:0/1175692139 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+153237 (secure 0 0 0) 0x7fdcc8016030 con 0x7fdcd806ad60 2024-02-20T12:51:02.293 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:51:02.292+0000 7fdcde748700 1 -- 172.21.15.64:0/1175692139 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) v1 -- 0x7fdcd81020f0 con 0x7fdcd806ad60 2024-02-20T12:51:02.293 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:51:02.292+0000 7fdcd4ff9700 1 -- 172.21.15.64:0/1175692139 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v10)=0 v10) v1 ==== 127+0+0 (secure 0 0 0) 0x7fdcd81020f0 con 0x7fdcd806ad60 2024-02-20T12:51:02.296 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:51:02.294+0000 7fdcde748700 1 -- 172.21.15.64:0/1175692139 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fdcc4031090 msgr2=0x7fdcc4033550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:51:02.296 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:51:02.294+0000 7fdcde748700 1 --2- 172.21.15.64:0/1175692139 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fdcc4031090 0x7fdcc4033550 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 rx=0x7fdccc0096a0 tx=0x7fdccc006b40).stop 2024-02-20T12:51:02.296 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:51:02.294+0000 7fdcde748700 1 -- 172.21.15.64:0/1175692139 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdcd806ad60 msgr2=0x7fdcd810cf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:51:02.296 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:51:02.294+0000 7fdcde748700 1 --2- 172.21.15.64:0/1175692139 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdcd806ad60 0x7fdcd810cf30 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 rx=0x7fdcc8006800 tx=0x7fdcc8006ee0).stop 2024-02-20T12:51:02.296 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:51:02.294+0000 7fdcde748700 1 -- 172.21.15.64:0/1175692139 shutdown_connections 2024-02-20T12:51:02.296 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:51:02.294+0000 7fdcde748700 1 --2- 172.21.15.64:0/1175692139 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fdcc4031090 0x7fdcc4033550 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:51:02.296 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:51:02.294+0000 7fdcde748700 1 --2- 172.21.15.64:0/1175692139 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdcd806ad60 0x7fdcd810cf30 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:51:02.296 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:51:02.294+0000 7fdcde748700 1 -- 172.21.15.64:0/1175692139 >> 172.21.15.64:0/1175692139 conn(0x7fdcd80f6680 msgr2=0x7fdcd80f80e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:51:02.297 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:51:02.294+0000 7fdcde748700 1 -- 172.21.15.64:0/1175692139 shutdown_connections 2024-02-20T12:51:02.297 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:51:02.294+0000 7fdcde748700 1 -- 172.21.15.64:0/1175692139 wait complete. 2024-02-20T12:51:03.590 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:51:03.576+0000 7f32de74c700 1 Processor -- start 2024-02-20T12:51:03.590 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:51:03.576+0000 7f32de74c700 1 -- start start 2024-02-20T12:51:03.590 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:51:03.576+0000 7f32de74c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f32d806ad00 0x7f32d80fecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-02-20T12:51:03.590 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:51:03.576+0000 7f32de74c700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32d80ff1b0 con 0x7f32d806ad00 2024-02-20T12:51:03.590 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:51:03.576+0000 7f32d7fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f32d806ad00 0x7f32d80fecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:51:03.590 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:51:03.577+0000 7f32d7fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f32d806ad00 0x7f32d80fecc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:46866/0 (socket says 172.21.15.64:46866) 2024-02-20T12:51:03.590 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:51:03.577+0000 7f32d7fff700 1 -- 172.21.15.64:0/4097401958 learned_addr learned my addr 172.21.15.64:0/4097401958 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-20T12:51:03.590 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:51:03.577+0000 7f32d7fff700 1 -- 172.21.15.64:0/4097401958 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32d80ff2f0 con 0x7f32d806ad00 2024-02-20T12:51:03.591 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:51:03.577+0000 7f32d7fff700 1 --2- 172.21.15.64:0/4097401958 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f32d806ad00 0x7f32d80fecc0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 rx=0x7f32c800b5f0 tx=0x7f32c8006ee0).ready entity=mon.0 client_cookie=306132d3b91ba08f server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:51:03.591 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:51:03.577+0000 7f32d6ffd700 1 -- 172.21.15.64:0/4097401958 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f32c800d600 con 0x7f32d806ad00 2024-02-20T12:51:03.591 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:51:03.577+0000 7f32d6ffd700 1 -- 172.21.15.64:0/4097401958 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(25 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f32c800dbf0 con 0x7f32d806ad00 2024-02-20T12:51:03.591 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:51:03.577+0000 7f32d6ffd700 1 -- 172.21.15.64:0/4097401958 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f32c8011bf0 con 0x7f32d806ad00 2024-02-20T12:51:03.591 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:51:03.578+0000 7f32de74c700 1 -- 172.21.15.64:0/4097401958 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f32d806ad00 msgr2=0x7f32d80fecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:51:03.591 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:51:03.578+0000 7f32de74c700 1 --2- 172.21.15.64:0/4097401958 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f32d806ad00 0x7f32d80fecc0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 rx=0x7f32c800b5f0 tx=0x7f32c8006ee0).stop 2024-02-20T12:51:03.591 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:51:03.578+0000 7f32de74c700 1 -- 172.21.15.64:0/4097401958 shutdown_connections 2024-02-20T12:51:03.591 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:51:03.578+0000 7f32de74c700 1 --2- 172.21.15.64:0/4097401958 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f32d806ad00 0x7f32d80fecc0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:51:03.591 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:51:03.578+0000 7f32de74c700 1 -- 172.21.15.64:0/4097401958 >> 172.21.15.64:0/4097401958 conn(0x7f32d80f6680 msgr2=0x7f32d80f8aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:51:03.591 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:51:03.578+0000 7f32de74c700 1 -- 172.21.15.64:0/4097401958 shutdown_connections 2024-02-20T12:51:03.591 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:51:03.578+0000 7f32de74c700 1 -- 172.21.15.64:0/4097401958 wait complete. 2024-02-20T12:51:03.591 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:51:03.578+0000 7f32de74c700 1 Processor -- start 2024-02-20T12:51:03.591 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:51:03.578+0000 7f32de74c700 1 -- start start 2024-02-20T12:51:03.592 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:51:03.579+0000 7f32de74c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f32d806ad00 0x7f32d810adc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:51:03.592 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:51:03.579+0000 7f32de74c700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32d810b2b0 con 0x7f32d806ad00 2024-02-20T12:51:03.592 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:51:03.579+0000 7f32d7fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f32d806ad00 0x7f32d810adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:51:03.592 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:51:03.579+0000 7f32d7fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f32d806ad00 0x7f32d810adc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:46872/0 (socket says 172.21.15.64:46872) 2024-02-20T12:51:03.592 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:51:03.579+0000 7f32d7fff700 1 -- 172.21.15.64:0/3419454160 learned_addr learned my addr 172.21.15.64:0/3419454160 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-20T12:51:03.592 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:51:03.579+0000 7f32d7fff700 1 -- 172.21.15.64:0/3419454160 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32c800b200 con 0x7f32d806ad00 2024-02-20T12:51:03.592 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:51:03.579+0000 7f32d7fff700 1 --2- 172.21.15.64:0/3419454160 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f32d806ad00 0x7f32d810adc0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 rx=0x7f32c8006800 tx=0x7f32c8006ee0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:51:03.592 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:51:03.579+0000 7f32d4ff9700 1 -- 172.21.15.64:0/3419454160 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f32c800dec0 con 0x7f32d806ad00 2024-02-20T12:51:03.592 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:51:03.579+0000 7f32d4ff9700 1 -- 172.21.15.64:0/3419454160 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(25 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f32c801b8a0 con 0x7f32d806ad00 2024-02-20T12:51:03.592 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:51:03.579+0000 7f32de74c700 1 -- 172.21.15.64:0/3419454160 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32d810b4b0 con 0x7f32d806ad00 2024-02-20T12:51:03.592 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:51:03.579+0000 7f32d4ff9700 1 -- 172.21.15.64:0/3419454160 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f32c801a690 con 0x7f32d806ad00 2024-02-20T12:51:03.592 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:51:03.579+0000 7f32de74c700 1 -- 172.21.15.64:0/3419454160 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32d810c0f0 con 0x7f32d806ad00 2024-02-20T12:51:03.592 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:51:03.580+0000 7f32d4ff9700 1 -- 172.21.15.64:0/3419454160 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 37151+0+0 (secure 0 0 0) 0x7f32c802b070 con 0x7f32d806ad00 2024-02-20T12:51:03.593 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:51:03.580+0000 7f32de74c700 1 -- 172.21.15.64:0/3419454160 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32d810b940 con 0x7f32d806ad00 2024-02-20T12:51:03.593 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:51:03.580+0000 7f32d4ff9700 1 --2- 172.21.15.64:0/3419454160 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f32c40310e0 0x7f32c40335a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:51:03.593 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:51:03.580+0000 7f32d4ff9700 1 -- 172.21.15.64:0/3419454160 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1065+0+0 (secure 0 0 0) 0x7f32c8042d00 con 0x7f32d806ad00 2024-02-20T12:51:03.593 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:51:03.580+0000 7f32d77fe700 1 --2- 172.21.15.64:0/3419454160 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f32c40310e0 0x7f32c40335a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:51:03.593 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:51:03.581+0000 7f32d77fe700 1 --2- 172.21.15.64:0/3419454160 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f32c40310e0 0x7f32c40335a0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 rx=0x7f32cc0096a0 tx=0x7f32cc006b40).ready entity=mgr.14156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:51:03.593 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:51:03.585+0000 7f32d4ff9700 1 -- 172.21.15.64:0/3419454160 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+153237 (secure 0 0 0) 0x7f32c801ba10 con 0x7f32d806ad00 2024-02-20T12:51:03.832 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:51:03.830+0000 7f32de74c700 1 -- 172.21.15.64:0/3419454160 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) v1 -- 0x7f32d8100040 con 0x7f32d806ad00 2024-02-20T12:51:03.834 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:51:03.833+0000 7f32d4ff9700 1 -- 172.21.15.64:0/3419454160 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v31)=0 set mgr/dashboard/cluster/status v31) v1 ==== 153+0+0 (secure 0 0 0) 0x7f32c801f020 con 0x7f32d806ad00 2024-02-20T12:51:03.836 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:51:03.834+0000 7f32de74c700 1 -- 172.21.15.64:0/3419454160 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f32c40310e0 msgr2=0x7f32c40335a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:51:03.836 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:51:03.834+0000 7f32de74c700 1 --2- 172.21.15.64:0/3419454160 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f32c40310e0 0x7f32c40335a0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 rx=0x7f32cc0096a0 tx=0x7f32cc006b40).stop 2024-02-20T12:51:03.836 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:51:03.834+0000 7f32de74c700 1 -- 172.21.15.64:0/3419454160 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f32d806ad00 msgr2=0x7f32d810adc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:51:03.837 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:51:03.834+0000 7f32de74c700 1 --2- 172.21.15.64:0/3419454160 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f32d806ad00 0x7f32d810adc0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 rx=0x7f32c8006800 tx=0x7f32c8006ee0).stop 2024-02-20T12:51:03.837 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:51:03.835+0000 7f32de74c700 1 -- 172.21.15.64:0/3419454160 shutdown_connections 2024-02-20T12:51:03.837 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:51:03.835+0000 7f32de74c700 1 --2- 172.21.15.64:0/3419454160 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f32c40310e0 0x7f32c40335a0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:51:03.837 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:51:03.835+0000 7f32de74c700 1 --2- 172.21.15.64:0/3419454160 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f32d806ad00 0x7f32d810adc0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:51:03.837 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:51:03.835+0000 7f32de74c700 1 -- 172.21.15.64:0/3419454160 >> 172.21.15.64:0/3419454160 conn(0x7f32d80f6680 msgr2=0x7f32d80f8150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:51:03.837 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:51:03.835+0000 7f32de74c700 1 -- 172.21.15.64:0/3419454160 shutdown_connections 2024-02-20T12:51:03.837 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: 2024-02-20T12:51:03.835+0000 7f32de74c700 1 -- 172.21.15.64:0/3419454160 wait complete. 2024-02-20T12:51:03.837 INFO:teuthology.orchestra.run.smithi064.stderr:/usr/bin/ceph: set mgr/dashboard/cluster/status 2024-02-20T12:51:04.600 INFO:teuthology.orchestra.run.smithi064.stderr:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-02-20T12:51:04.601 INFO:teuthology.orchestra.run.smithi064.stderr: 2024-02-20T12:51:04.601 INFO:teuthology.orchestra.run.smithi064.stderr: sudo /home/ubuntu/cephtest/cephadm shell --fsid 680ee126-cfee-11ee-95bb-87774f69a715 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-02-20T12:51:04.601 INFO:teuthology.orchestra.run.smithi064.stderr: 2024-02-20T12:51:04.601 INFO:teuthology.orchestra.run.smithi064.stderr:Or, if you are only running a single cluster on this host: 2024-02-20T12:51:04.601 INFO:teuthology.orchestra.run.smithi064.stderr: 2024-02-20T12:51:04.601 INFO:teuthology.orchestra.run.smithi064.stderr: sudo /home/ubuntu/cephtest/cephadm shell 2024-02-20T12:51:04.601 INFO:teuthology.orchestra.run.smithi064.stderr: 2024-02-20T12:51:04.601 INFO:teuthology.orchestra.run.smithi064.stderr:Please consider enabling telemetry to help improve Ceph: 2024-02-20T12:51:04.601 INFO:teuthology.orchestra.run.smithi064.stderr: 2024-02-20T12:51:04.601 INFO:teuthology.orchestra.run.smithi064.stderr: ceph telemetry on 2024-02-20T12:51:04.601 INFO:teuthology.orchestra.run.smithi064.stderr: 2024-02-20T12:51:04.601 INFO:teuthology.orchestra.run.smithi064.stderr:For more information see: 2024-02-20T12:51:04.601 INFO:teuthology.orchestra.run.smithi064.stderr: 2024-02-20T12:51:04.602 INFO:teuthology.orchestra.run.smithi064.stderr: https://docs.ceph.com/en/pacific/mgr/telemetry/ 2024-02-20T12:51:04.602 INFO:teuthology.orchestra.run.smithi064.stderr: 2024-02-20T12:51:04.602 INFO:teuthology.orchestra.run.smithi064.stderr:Bootstrap complete. 2024-02-20T12:51:04.643 INFO:tasks.cephadm:Fetching config... 2024-02-20T12:51:04.644 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-02-20T12:51:04.644 DEBUG:teuthology.orchestra.run.smithi064:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-02-20T12:51:04.705 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-02-20T12:51:04.705 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-02-20T12:51:04.705 DEBUG:teuthology.orchestra.run.smithi064:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-02-20T12:51:04.764 INFO:tasks.cephadm:Fetching mon keyring... 2024-02-20T12:51:04.764 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-02-20T12:51:04.764 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd if=/var/lib/ceph/680ee126-cfee-11ee-95bb-87774f69a715/mon.smithi064/keyring of=/dev/stdout 2024-02-20T12:51:04.834 INFO:tasks.cephadm:Fetching pub ssh key... 2024-02-20T12:51:04.834 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-02-20T12:51:04.834 DEBUG:teuthology.orchestra.run.smithi064:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-02-20T12:51:04.893 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-02-20T12:51:04.893 DEBUG:teuthology.orchestra.run.smithi064:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDKOQCSumDj3118UsALF+LqZu5FOGmhIFdUrATcTf8tvctNd0xnOBj3cDTxFUzyVjPw599qypes3HlfCkN+PUvf6tXuoEOigfi07scE0UkkxmCd9d5+cXmXw+ugH+0e75mpJaX04hcG02wrHx5YjHpKz6K+XgWpW+xizjFJO2/g4ceNUBfW93qKexpstyhP6CwGm7ywxreZcFW+QUiEWVXgPYrzMvDUsdqN+5P6KjVvEGq5jECJEstco/r1MZgOHCFAr3v7gt7BYav0pFg1aQx6+NHExOMtq07OA15dD3MB0wk+k0ypAqKM3eV3M2nV/kK+UXjuhAelnDe1LTAD/P/opKtmBUSzh+S14L6VarVdF3LaX1+THnOMNHkmR6BoU+BiQ1hz38p/lDBlv2/0HLbZbbLSnV8Xzl481F90qiNBYblm+wbFFxCXi6woVETn+J5yikpC3rHKOa0RnDIouxA4V698sI42MY77qWmwWE8OvPYnGsXkuNJkCacCMTzNd0s= ceph-680ee126-cfee-11ee-95bb-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-02-20T12:51:04.960 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:51:04 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:51:03.833979+0000 mon.smithi064 (mon.0) 95 : audit [INF] from='client.? 172.21.15.64:0/3419454160' entity='client.admin' 2024-02-20T12:51:04.974 INFO:teuthology.orchestra.run.smithi064.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDKOQCSumDj3118UsALF+LqZu5FOGmhIFdUrATcTf8tvctNd0xnOBj3cDTxFUzyVjPw599qypes3HlfCkN+PUvf6tXuoEOigfi07scE0UkkxmCd9d5+cXmXw+ugH+0e75mpJaX04hcG02wrHx5YjHpKz6K+XgWpW+xizjFJO2/g4ceNUBfW93qKexpstyhP6CwGm7ywxreZcFW+QUiEWVXgPYrzMvDUsdqN+5P6KjVvEGq5jECJEstco/r1MZgOHCFAr3v7gt7BYav0pFg1aQx6+NHExOMtq07OA15dD3MB0wk+k0ypAqKM3eV3M2nV/kK+UXjuhAelnDe1LTAD/P/opKtmBUSzh+S14L6VarVdF3LaX1+THnOMNHkmR6BoU+BiQ1hz38p/lDBlv2/0HLbZbbLSnV8Xzl481F90qiNBYblm+wbFFxCXi6woVETn+J5yikpC3rHKOa0RnDIouxA4V698sI42MY77qWmwWE8OvPYnGsXkuNJkCacCMTzNd0s= ceph-680ee126-cfee-11ee-95bb-87774f69a715 2024-02-20T12:51:04.988 DEBUG:teuthology.orchestra.run.smithi071:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDKOQCSumDj3118UsALF+LqZu5FOGmhIFdUrATcTf8tvctNd0xnOBj3cDTxFUzyVjPw599qypes3HlfCkN+PUvf6tXuoEOigfi07scE0UkkxmCd9d5+cXmXw+ugH+0e75mpJaX04hcG02wrHx5YjHpKz6K+XgWpW+xizjFJO2/g4ceNUBfW93qKexpstyhP6CwGm7ywxreZcFW+QUiEWVXgPYrzMvDUsdqN+5P6KjVvEGq5jECJEstco/r1MZgOHCFAr3v7gt7BYav0pFg1aQx6+NHExOMtq07OA15dD3MB0wk+k0ypAqKM3eV3M2nV/kK+UXjuhAelnDe1LTAD/P/opKtmBUSzh+S14L6VarVdF3LaX1+THnOMNHkmR6BoU+BiQ1hz38p/lDBlv2/0HLbZbbLSnV8Xzl481F90qiNBYblm+wbFFxCXi6woVETn+J5yikpC3rHKOa0RnDIouxA4V698sI42MY77qWmwWE8OvPYnGsXkuNJkCacCMTzNd0s= ceph-680ee126-cfee-11ee-95bb-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-02-20T12:51:05.033 INFO:teuthology.orchestra.run.smithi071.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDKOQCSumDj3118UsALF+LqZu5FOGmhIFdUrATcTf8tvctNd0xnOBj3cDTxFUzyVjPw599qypes3HlfCkN+PUvf6tXuoEOigfi07scE0UkkxmCd9d5+cXmXw+ugH+0e75mpJaX04hcG02wrHx5YjHpKz6K+XgWpW+xizjFJO2/g4ceNUBfW93qKexpstyhP6CwGm7ywxreZcFW+QUiEWVXgPYrzMvDUsdqN+5P6KjVvEGq5jECJEstco/r1MZgOHCFAr3v7gt7BYav0pFg1aQx6+NHExOMtq07OA15dD3MB0wk+k0ypAqKM3eV3M2nV/kK+UXjuhAelnDe1LTAD/P/opKtmBUSzh+S14L6VarVdF3LaX1+THnOMNHkmR6BoU+BiQ1hz38p/lDBlv2/0HLbZbbLSnV8Xzl481F90qiNBYblm+wbFFxCXi6woVETn+J5yikpC3rHKOa0RnDIouxA4V698sI42MY77qWmwWE8OvPYnGsXkuNJkCacCMTzNd0s= ceph-680ee126-cfee-11ee-95bb-87774f69a715 2024-02-20T12:51:05.045 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/daemon-base:latest-pacific shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 680ee126-cfee-11ee-95bb-87774f69a715 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-02-20T12:51:07.666 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-20T12:51:07.664+0000 7f255c0b9700 1 -- 172.21.15.64:0/3223624086 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f25540f8790 msgr2=0x7f25540f8bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:51:07.667 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-20T12:51:07.664+0000 7f255c0b9700 1 --2- 172.21.15.64:0/3223624086 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f25540f8790 0x7f25540f8bb0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 rx=0x7f2550004760 tx=0x7f2550009ad0).stop 2024-02-20T12:51:07.667 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-20T12:51:07.664+0000 7f255c0b9700 1 -- 172.21.15.64:0/3223624086 shutdown_connections 2024-02-20T12:51:07.667 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-20T12:51:07.664+0000 7f255c0b9700 1 --2- 172.21.15.64:0/3223624086 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f25540f8790 0x7f25540f8bb0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:51:07.667 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-20T12:51:07.664+0000 7f255c0b9700 1 -- 172.21.15.64:0/3223624086 >> 172.21.15.64:0/3223624086 conn(0x7f25540f41c0 msgr2=0x7f25540f65b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:51:07.667 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-20T12:51:07.664+0000 7f255c0b9700 1 -- 172.21.15.64:0/3223624086 shutdown_connections 2024-02-20T12:51:07.667 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-20T12:51:07.664+0000 7f255c0b9700 1 -- 172.21.15.64:0/3223624086 wait complete. 2024-02-20T12:51:07.667 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-20T12:51:07.664+0000 7f255c0b9700 1 Processor -- start 2024-02-20T12:51:07.667 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-20T12:51:07.664+0000 7f255c0b9700 1 -- start start 2024-02-20T12:51:07.667 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-20T12:51:07.665+0000 7f255c0b9700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f25540f8790 0x7f25541089e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:51:07.667 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-20T12:51:07.665+0000 7f255c0b9700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2554108ed0 con 0x7f25540f8790 2024-02-20T12:51:07.667 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-20T12:51:07.665+0000 7f2559e55700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f25540f8790 0x7f25541089e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:51:07.668 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-20T12:51:07.665+0000 7f2559e55700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f25540f8790 0x7f25541089e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:46888/0 (socket says 172.21.15.64:46888) 2024-02-20T12:51:07.668 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-20T12:51:07.665+0000 7f2559e55700 1 -- 172.21.15.64:0/2250593671 learned_addr learned my addr 172.21.15.64:0/2250593671 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-20T12:51:07.668 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-20T12:51:07.665+0000 7f2559e55700 1 -- 172.21.15.64:0/2250593671 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2550009d70 con 0x7f25540f8790 2024-02-20T12:51:07.668 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-20T12:51:07.665+0000 7f2559e55700 1 --2- 172.21.15.64:0/2250593671 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f25540f8790 0x7f25541089e0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 rx=0x7f2550013f10 tx=0x7f2550009ad0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:51:07.668 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-20T12:51:07.665+0000 7f254a7fc700 1 -- 172.21.15.64:0/2250593671 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f255000e040 con 0x7f25540f8790 2024-02-20T12:51:07.668 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-20T12:51:07.665+0000 7f254a7fc700 1 -- 172.21.15.64:0/2250593671 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(25 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f2550007df0 con 0x7f25540f8790 2024-02-20T12:51:07.668 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-20T12:51:07.665+0000 7f255c0b9700 1 -- 172.21.15.64:0/2250593671 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25541090d0 con 0x7f25540f8790 2024-02-20T12:51:07.668 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-20T12:51:07.665+0000 7f254a7fc700 1 -- 172.21.15.64:0/2250593671 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2550014d60 con 0x7f25540f8790 2024-02-20T12:51:07.668 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-20T12:51:07.665+0000 7f255c0b9700 1 -- 172.21.15.64:0/2250593671 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2554109560 con 0x7f25540f8790 2024-02-20T12:51:07.669 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-20T12:51:07.666+0000 7f255c0b9700 1 -- 172.21.15.64:0/2250593671 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2554045bb0 con 0x7f25540f8790 2024-02-20T12:51:07.669 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-20T12:51:07.667+0000 7f254a7fc700 1 -- 172.21.15.64:0/2250593671 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 37151+0+0 (secure 0 0 0) 0x7f2550014520 con 0x7f25540f8790 2024-02-20T12:51:07.669 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-20T12:51:07.667+0000 7f254a7fc700 1 --2- 172.21.15.64:0/2250593671 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f2540031090 0x7f2540033550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:51:07.669 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-20T12:51:07.667+0000 7f254a7fc700 1 -- 172.21.15.64:0/2250593671 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1065+0+0 (secure 0 0 0) 0x7f25500426f0 con 0x7f25540f8790 2024-02-20T12:51:07.669 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-20T12:51:07.667+0000 7f2559654700 1 --2- 172.21.15.64:0/2250593671 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f2540031090 0x7f2540033550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:51:07.669 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-20T12:51:07.667+0000 7f2559654700 1 --2- 172.21.15.64:0/2250593671 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f2540031090 0x7f2540033550 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 rx=0x7f25440096a0 tx=0x7f2544006b40).ready entity=mgr.14156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:51:07.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-20T12:51:07.671+0000 7f254a7fc700 1 -- 172.21.15.64:0/2250593671 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+153237 (secure 0 0 0) 0x7f2550012b60 con 0x7f25540f8790 2024-02-20T12:51:07.833 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-20T12:51:07.832+0000 7f255c0b9700 1 -- 172.21.15.64:0/2250593671 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) v1 -- 0x7f25540fe280 con 0x7f25540f8790 2024-02-20T12:51:07.836 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-20T12:51:07.835+0000 7f254a7fc700 1 -- 172.21.15.64:0/2250593671 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v11)=0 v11) v1 ==== 125+0+0 (secure 0 0 0) 0x7f2550012330 con 0x7f25540f8790 2024-02-20T12:51:07.838 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-20T12:51:07.837+0000 7f255c0b9700 1 -- 172.21.15.64:0/2250593671 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f2540031090 msgr2=0x7f2540033550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:51:07.838 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-20T12:51:07.837+0000 7f255c0b9700 1 --2- 172.21.15.64:0/2250593671 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f2540031090 0x7f2540033550 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 rx=0x7f25440096a0 tx=0x7f2544006b40).stop 2024-02-20T12:51:07.838 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-20T12:51:07.837+0000 7f255c0b9700 1 -- 172.21.15.64:0/2250593671 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f25540f8790 msgr2=0x7f25541089e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:51:07.838 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-20T12:51:07.837+0000 7f255c0b9700 1 --2- 172.21.15.64:0/2250593671 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f25540f8790 0x7f25541089e0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 rx=0x7f2550013f10 tx=0x7f2550009ad0).stop 2024-02-20T12:51:07.839 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-20T12:51:07.837+0000 7f255c0b9700 1 -- 172.21.15.64:0/2250593671 shutdown_connections 2024-02-20T12:51:07.839 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-20T12:51:07.837+0000 7f255c0b9700 1 --2- 172.21.15.64:0/2250593671 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f2540031090 0x7f2540033550 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:51:07.839 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-20T12:51:07.837+0000 7f255c0b9700 1 --2- 172.21.15.64:0/2250593671 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f25540f8790 0x7f25541089e0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:51:07.839 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-20T12:51:07.837+0000 7f255c0b9700 1 -- 172.21.15.64:0/2250593671 >> 172.21.15.64:0/2250593671 conn(0x7f25540f41c0 msgr2=0x7f25540f5cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:51:07.839 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-20T12:51:07.838+0000 7f255c0b9700 1 -- 172.21.15.64:0/2250593671 shutdown_connections 2024-02-20T12:51:07.839 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-20T12:51:07.838+0000 7f255c0b9700 1 -- 172.21.15.64:0/2250593671 wait complete. 2024-02-20T12:51:09.087 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-02-20T12:51:09.088 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/daemon-base:latest-pacific shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 680ee126-cfee-11ee-95bb-87774f69a715 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-02-20T12:51:09.116 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:51:08 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:51:07.836782+0000 mon.smithi064 (mon.0) 96 : audit [INF] from='client.? 172.21.15.64:0/2250593671' entity='client.admin' 2024-02-20T12:51:11.854 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-20T12:51:11.852+0000 7ff31724a700 1 -- 172.21.15.64:0/360359175 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff3100602a0 msgr2=0x7ff3100f9070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:51:11.854 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-20T12:51:11.852+0000 7ff31724a700 1 --2- 172.21.15.64:0/360359175 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff3100602a0 0x7ff3100f9070 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 rx=0x7ff30c009fb0 tx=0x7ff30c009b30).stop 2024-02-20T12:51:11.854 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-20T12:51:11.852+0000 7ff31724a700 1 -- 172.21.15.64:0/360359175 shutdown_connections 2024-02-20T12:51:11.854 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-20T12:51:11.852+0000 7ff31724a700 1 --2- 172.21.15.64:0/360359175 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff3100602a0 0x7ff3100f9070 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:51:11.854 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-20T12:51:11.852+0000 7ff31724a700 1 -- 172.21.15.64:0/360359175 >> 172.21.15.64:0/360359175 conn(0x7ff3100f0a30 msgr2=0x7ff3100f2e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:51:11.854 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-20T12:51:11.852+0000 7ff31724a700 1 -- 172.21.15.64:0/360359175 shutdown_connections 2024-02-20T12:51:11.854 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-20T12:51:11.852+0000 7ff31724a700 1 -- 172.21.15.64:0/360359175 wait complete. 2024-02-20T12:51:11.856 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-20T12:51:11.853+0000 7ff31724a700 1 Processor -- start 2024-02-20T12:51:11.856 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-20T12:51:11.853+0000 7ff31724a700 1 -- start start 2024-02-20T12:51:11.856 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-20T12:51:11.854+0000 7ff31724a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff3100602a0 0x7ff3101065c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:51:11.856 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-20T12:51:11.854+0000 7ff31724a700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff310106ab0 con 0x7ff3100602a0 2024-02-20T12:51:11.856 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-20T12:51:11.854+0000 7ff316248700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff3100602a0 0x7ff3101065c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:51:11.856 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-20T12:51:11.854+0000 7ff316248700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff3100602a0 0x7ff3101065c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:37978/0 (socket says 172.21.15.64:37978) 2024-02-20T12:51:11.856 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-20T12:51:11.854+0000 7ff316248700 1 -- 172.21.15.64:0/2191723363 learned_addr learned my addr 172.21.15.64:0/2191723363 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-20T12:51:11.856 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-20T12:51:11.854+0000 7ff316248700 1 -- 172.21.15.64:0/2191723363 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff30c005040 con 0x7ff3100602a0 2024-02-20T12:51:11.856 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-20T12:51:11.854+0000 7ff316248700 1 --2- 172.21.15.64:0/2191723363 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff3100602a0 0x7ff3101065c0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 rx=0x7ff30c000c00 tx=0x7ff30c00b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:51:11.856 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-20T12:51:11.854+0000 7ff306ffd700 1 -- 172.21.15.64:0/2191723363 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff30c0095f0 con 0x7ff3100602a0 2024-02-20T12:51:11.856 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-20T12:51:11.854+0000 7ff306ffd700 1 -- 172.21.15.64:0/2191723363 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(25 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7ff30c007e50 con 0x7ff3100602a0 2024-02-20T12:51:11.856 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-20T12:51:11.854+0000 7ff306ffd700 1 -- 172.21.15.64:0/2191723363 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff30c014e70 con 0x7ff3100602a0 2024-02-20T12:51:11.856 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-20T12:51:11.854+0000 7ff31724a700 1 -- 172.21.15.64:0/2191723363 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff310106cb0 con 0x7ff3100602a0 2024-02-20T12:51:11.857 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-20T12:51:11.854+0000 7ff31724a700 1 -- 172.21.15.64:0/2191723363 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff310107140 con 0x7ff3100602a0 2024-02-20T12:51:11.857 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-20T12:51:11.855+0000 7ff306ffd700 1 -- 172.21.15.64:0/2191723363 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 37151+0+0 (secure 0 0 0) 0x7ff30c025070 con 0x7ff3100602a0 2024-02-20T12:51:11.858 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-20T12:51:11.855+0000 7ff306ffd700 1 --2- 172.21.15.64:0/2191723363 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7ff2fc0310e0 0x7ff2fc0335a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:51:11.858 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-20T12:51:11.855+0000 7ff31724a700 1 -- 172.21.15.64:0/2191723363 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff310045bb0 con 0x7ff3100602a0 2024-02-20T12:51:11.858 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-20T12:51:11.856+0000 7ff306ffd700 1 -- 172.21.15.64:0/2191723363 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1065+0+0 (secure 0 0 0) 0x7ff30c041d30 con 0x7ff3100602a0 2024-02-20T12:51:11.858 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-20T12:51:11.856+0000 7ff315a47700 1 --2- 172.21.15.64:0/2191723363 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7ff2fc0310e0 0x7ff2fc0335a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:51:11.858 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-20T12:51:11.856+0000 7ff315a47700 1 --2- 172.21.15.64:0/2191723363 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7ff2fc0310e0 0x7ff2fc0335a0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 rx=0x7ff3000096a0 tx=0x7ff300006b40).ready entity=mgr.14156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:51:11.861 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-20T12:51:11.860+0000 7ff306ffd700 1 -- 172.21.15.64:0/2191723363 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+153237 (secure 0 0 0) 0x7ff30c026350 con 0x7ff3100602a0 2024-02-20T12:51:12.055 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-20T12:51:12.054+0000 7ff31724a700 1 -- 172.21.15.64:0/2191723363 --> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) v1 -- 0x7ff3100f1c30 con 0x7ff2fc0310e0 2024-02-20T12:51:12.061 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-20T12:51:12.060+0000 7ff306ffd700 1 -- 172.21.15.64:0/2191723363 <== mgr.14156 v2:172.21.15.64:6800/1408391390 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7ff3100f1c30 con 0x7ff2fc0310e0 2024-02-20T12:51:12.063 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-20T12:51:12.062+0000 7ff31724a700 1 -- 172.21.15.64:0/2191723363 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7ff2fc0310e0 msgr2=0x7ff2fc0335a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:51:12.063 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-20T12:51:12.062+0000 7ff31724a700 1 --2- 172.21.15.64:0/2191723363 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7ff2fc0310e0 0x7ff2fc0335a0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 rx=0x7ff3000096a0 tx=0x7ff300006b40).stop 2024-02-20T12:51:12.063 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-20T12:51:12.062+0000 7ff31724a700 1 -- 172.21.15.64:0/2191723363 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff3100602a0 msgr2=0x7ff3101065c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:51:12.063 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-20T12:51:12.062+0000 7ff31724a700 1 --2- 172.21.15.64:0/2191723363 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff3100602a0 0x7ff3101065c0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 rx=0x7ff30c000c00 tx=0x7ff30c00b040).stop 2024-02-20T12:51:12.063 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-20T12:51:12.062+0000 7ff31724a700 1 -- 172.21.15.64:0/2191723363 shutdown_connections 2024-02-20T12:51:12.063 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-20T12:51:12.062+0000 7ff31724a700 1 --2- 172.21.15.64:0/2191723363 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7ff2fc0310e0 0x7ff2fc0335a0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:51:12.063 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-20T12:51:12.062+0000 7ff31724a700 1 --2- 172.21.15.64:0/2191723363 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff3100602a0 0x7ff3101065c0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:51:12.064 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-20T12:51:12.062+0000 7ff31724a700 1 -- 172.21.15.64:0/2191723363 >> 172.21.15.64:0/2191723363 conn(0x7ff3100f0a30 msgr2=0x7ff3100f1520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:51:12.064 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-20T12:51:12.062+0000 7ff31724a700 1 -- 172.21.15.64:0/2191723363 shutdown_connections 2024-02-20T12:51:12.064 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-20T12:51:12.062+0000 7ff31724a700 1 -- 172.21.15.64:0/2191723363 wait complete. 2024-02-20T12:51:12.472 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:51:12 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:51:11.321504+0000 mon.smithi064 (mon.0) 97 : audit [INF] from='mgr.14156 172.21.15.64:0/3284974785' entity='mgr.smithi064.vriyor' 2024-02-20T12:51:12.472 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:51:12 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:51:11.678668+0000 mon.smithi064 (mon.0) 98 : audit [INF] from='mgr.14156 172.21.15.64:0/3284974785' entity='mgr.smithi064.vriyor' cmd=[{"prefix": "config rm", "who": "osd/host:smithi064", "name": "osd_memory_target"}]: dispatch 2024-02-20T12:51:12.472 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:51:12 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:51:11.684439+0000 mon.smithi064 (mon.0) 99 : audit [INF] from='mgr.14156 172.21.15.64:0/3284974785' entity='mgr.smithi064.vriyor' 2024-02-20T12:51:12.472 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:51:12 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:51:12.061292+0000 mon.smithi064 (mon.0) 100 : audit [INF] from='mgr.14156 172.21.15.64:0/3284974785' entity='mgr.smithi064.vriyor' 2024-02-20T12:51:12.797 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi071 2024-02-20T12:51:12.797 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-02-20T12:51:12.797 DEBUG:teuthology.orchestra.run.smithi071:> dd of=/etc/ceph/ceph.conf 2024-02-20T12:51:12.819 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-02-20T12:51:12.819 DEBUG:teuthology.orchestra.run.smithi071:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-02-20T12:51:12.881 INFO:tasks.cephadm:Adding host smithi071 to orchestrator... 2024-02-20T12:51:12.881 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/daemon-base:latest-pacific shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 680ee126-cfee-11ee-95bb-87774f69a715 -- ceph orch host add smithi071 2024-02-20T12:51:13.644 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:51:13 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cephadm 2024-02-20T12:51:11.691485+0000 mgr.smithi064.vriyor (mgr.14156) 5 : cephadm [INF] Deploying daemon alertmanager.smithi064 on smithi064 2024-02-20T12:51:13.644 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:51:13 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:51:12.056069+0000 mgr.smithi064.vriyor (mgr.14156) 6 : audit [DBG] from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-02-20T12:51:15.101 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-20T12:51:15.098+0000 7fe1809fe700 1 -- 172.21.15.64:0/2010913788 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe17c0f58a0 msgr2=0x7fe17c0f5cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:51:15.101 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-20T12:51:15.098+0000 7fe1809fe700 1 --2- 172.21.15.64:0/2010913788 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe17c0f58a0 0x7fe17c0f5cc0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 rx=0x7fe16c009fb0 tx=0x7fe16c009b30).stop 2024-02-20T12:51:15.102 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-20T12:51:15.098+0000 7fe1809fe700 1 -- 172.21.15.64:0/2010913788 shutdown_connections 2024-02-20T12:51:15.102 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-20T12:51:15.098+0000 7fe1809fe700 1 --2- 172.21.15.64:0/2010913788 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe17c0f58a0 0x7fe17c0f5cc0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:51:15.102 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-20T12:51:15.098+0000 7fe1809fe700 1 -- 172.21.15.64:0/2010913788 >> 172.21.15.64:0/2010913788 conn(0x7fe17c0f0190 msgr2=0x7fe17c0f25d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:51:15.102 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-20T12:51:15.098+0000 7fe1809fe700 1 -- 172.21.15.64:0/2010913788 shutdown_connections 2024-02-20T12:51:15.102 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-20T12:51:15.099+0000 7fe1809fe700 1 -- 172.21.15.64:0/2010913788 wait complete. 2024-02-20T12:51:15.102 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-20T12:51:15.099+0000 7fe1809fe700 1 Processor -- start 2024-02-20T12:51:15.102 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-20T12:51:15.099+0000 7fe1809fe700 1 -- start start 2024-02-20T12:51:15.102 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-20T12:51:15.099+0000 7fe1809fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe17c0f58a0 0x7fe17c0fd2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:51:15.102 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-20T12:51:15.099+0000 7fe1809fe700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe17c0fb730 con 0x7fe17c0f58a0 2024-02-20T12:51:15.102 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-20T12:51:15.100+0000 7fe17ad9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe17c0f58a0 0x7fe17c0fd2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:51:15.102 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-20T12:51:15.100+0000 7fe17ad9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe17c0f58a0 0x7fe17c0fd2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:38002/0 (socket says 172.21.15.64:38002) 2024-02-20T12:51:15.102 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-20T12:51:15.100+0000 7fe17ad9d700 1 -- 172.21.15.64:0/2956046034 learned_addr learned my addr 172.21.15.64:0/2956046034 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-20T12:51:15.102 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-20T12:51:15.100+0000 7fe17ad9d700 1 -- 172.21.15.64:0/2956046034 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe16c005040 con 0x7fe17c0f58a0 2024-02-20T12:51:15.102 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-20T12:51:15.100+0000 7fe17ad9d700 1 --2- 172.21.15.64:0/2956046034 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe17c0f58a0 0x7fe17c0fd2b0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 rx=0x7fe16c0089f0 tx=0x7fe16c00b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:51:15.102 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-20T12:51:15.100+0000 7fe1737fe700 1 -- 172.21.15.64:0/2956046034 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe16c009320 con 0x7fe17c0f58a0 2024-02-20T12:51:15.103 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-20T12:51:15.100+0000 7fe1737fe700 1 -- 172.21.15.64:0/2956046034 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(25 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7fe16c013ec0 con 0x7fe17c0f58a0 2024-02-20T12:51:15.103 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-20T12:51:15.100+0000 7fe1809fe700 1 -- 172.21.15.64:0/2956046034 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe17c0fb8d0 con 0x7fe17c0f58a0 2024-02-20T12:51:15.103 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-20T12:51:15.100+0000 7fe1737fe700 1 -- 172.21.15.64:0/2956046034 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe16c014d30 con 0x7fe17c0f58a0 2024-02-20T12:51:15.103 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-20T12:51:15.100+0000 7fe1809fe700 1 -- 172.21.15.64:0/2956046034 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe17c0fbd60 con 0x7fe17c0f58a0 2024-02-20T12:51:15.104 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-20T12:51:15.101+0000 7fe1737fe700 1 -- 172.21.15.64:0/2956046034 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 37151+0+0 (secure 0 0 0) 0x7fe16c0263a0 con 0x7fe17c0f58a0 2024-02-20T12:51:15.104 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-20T12:51:15.101+0000 7fe1737fe700 1 --2- 172.21.15.64:0/2956046034 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fe168031090 0x7fe168033550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:51:15.104 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-20T12:51:15.101+0000 7fe1737fe700 1 -- 172.21.15.64:0/2956046034 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1065+0+0 (secure 0 0 0) 0x7fe16c0095f0 con 0x7fe17c0f58a0 2024-02-20T12:51:15.104 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-20T12:51:15.101+0000 7fe17a59c700 1 --2- 172.21.15.64:0/2956046034 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fe168031090 0x7fe168033550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:51:15.104 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-20T12:51:15.101+0000 7fe1809fe700 1 -- 172.21.15.64:0/2956046034 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe17c0591e0 con 0x7fe17c0f58a0 2024-02-20T12:51:15.104 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-20T12:51:15.102+0000 7fe17a59c700 1 --2- 172.21.15.64:0/2956046034 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fe168031090 0x7fe168033550 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 rx=0x7fe1640096a0 tx=0x7fe164006b40).ready entity=mgr.14156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:51:15.106 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-20T12:51:15.106+0000 7fe1737fe700 1 -- 172.21.15.64:0/2956046034 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+153237 (secure 0 0 0) 0x7fe16c0266d0 con 0x7fe17c0f58a0 2024-02-20T12:51:15.271 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-20T12:51:15.270+0000 7fe1809fe700 1 -- 172.21.15.64:0/2956046034 --> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi071", "target": ["mon-mgr", ""]}) v1 -- 0x7fe17c0fc420 con 0x7fe168031090 2024-02-20T12:51:15.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:51:15 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:51:13.703093+0000 mgr.smithi064.vriyor (mgr.14156) 7 : cluster [DBG] pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:51:15.900 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-20T12:51:15.898+0000 7fe1737fe700 1 -- 172.21.15.64:0/2956046034 <== mgr.14156 v2:172.21.15.64:6800/1408391390 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+48 (secure 0 0 0) 0x7fe17c0fc420 con 0x7fe168031090 2024-02-20T12:51:15.900 INFO:teuthology.orchestra.run.smithi064.stdout:Added host 'smithi071' with addr '172.21.15.71' 2024-02-20T12:51:15.901 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-20T12:51:15.900+0000 7fe1809fe700 1 -- 172.21.15.64:0/2956046034 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fe168031090 msgr2=0x7fe168033550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:51:15.902 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-20T12:51:15.900+0000 7fe1809fe700 1 --2- 172.21.15.64:0/2956046034 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fe168031090 0x7fe168033550 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 rx=0x7fe1640096a0 tx=0x7fe164006b40).stop 2024-02-20T12:51:15.902 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-20T12:51:15.900+0000 7fe1809fe700 1 -- 172.21.15.64:0/2956046034 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe17c0f58a0 msgr2=0x7fe17c0fd2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:51:15.902 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-20T12:51:15.900+0000 7fe1809fe700 1 --2- 172.21.15.64:0/2956046034 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe17c0f58a0 0x7fe17c0fd2b0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 rx=0x7fe16c0089f0 tx=0x7fe16c00b040).stop 2024-02-20T12:51:15.902 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-20T12:51:15.900+0000 7fe1809fe700 1 -- 172.21.15.64:0/2956046034 shutdown_connections 2024-02-20T12:51:15.902 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-20T12:51:15.900+0000 7fe1809fe700 1 --2- 172.21.15.64:0/2956046034 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fe168031090 0x7fe168033550 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:51:15.902 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-20T12:51:15.900+0000 7fe1809fe700 1 --2- 172.21.15.64:0/2956046034 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe17c0f58a0 0x7fe17c0fd2b0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:51:15.902 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-20T12:51:15.900+0000 7fe1809fe700 1 -- 172.21.15.64:0/2956046034 >> 172.21.15.64:0/2956046034 conn(0x7fe17c0f0190 msgr2=0x7fe17c0f4e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:51:15.902 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-20T12:51:15.900+0000 7fe1809fe700 1 -- 172.21.15.64:0/2956046034 shutdown_connections 2024-02-20T12:51:15.902 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-20T12:51:15.901+0000 7fe1809fe700 1 -- 172.21.15.64:0/2956046034 wait complete. 2024-02-20T12:51:16.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:51:16 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:51:15.272155+0000 mgr.smithi064.vriyor (mgr.14156) 8 : audit [DBG] from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi071", "target": ["mon-mgr", ""]}]: dispatch 2024-02-20T12:51:16.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:51:16 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:51:15.899164+0000 mon.smithi064 (mon.0) 101 : audit [INF] from='mgr.14156 172.21.15.64:0/3284974785' entity='mgr.smithi064.vriyor' 2024-02-20T12:51:16.608 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/daemon-base:latest-pacific shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 680ee126-cfee-11ee-95bb-87774f69a715 -- ceph orch host ls --format=json 2024-02-20T12:51:17.719 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:51:17 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:51:15.703480+0000 mgr.smithi064.vriyor (mgr.14156) 9 : cluster [DBG] pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:51:17.719 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:51:17 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cephadm 2024-02-20T12:51:15.899480+0000 mgr.smithi064.vriyor (mgr.14156) 10 : cephadm [INF] Added host smithi071 2024-02-20T12:51:18.644 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-20T12:51:18.627+0000 7f784c485700 1 -- 172.21.15.64:0/2757148432 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f78440f6590 msgr2=0x7f78440f69b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:51:18.644 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-20T12:51:18.627+0000 7f784c485700 1 --2- 172.21.15.64:0/2757148432 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f78440f6590 0x7f78440f69b0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 rx=0x7f7840004760 tx=0x7f7840009ad0).stop 2024-02-20T12:51:18.644 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-20T12:51:18.627+0000 7f784c485700 1 -- 172.21.15.64:0/2757148432 shutdown_connections 2024-02-20T12:51:18.644 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-20T12:51:18.627+0000 7f784c485700 1 --2- 172.21.15.64:0/2757148432 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f78440f6590 0x7f78440f69b0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:51:18.644 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-20T12:51:18.628+0000 7f784c485700 1 -- 172.21.15.64:0/2757148432 >> 172.21.15.64:0/2757148432 conn(0x7f78440f1cd0 msgr2=0x7f78440f4110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:51:18.644 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-20T12:51:18.628+0000 7f784c485700 1 -- 172.21.15.64:0/2757148432 shutdown_connections 2024-02-20T12:51:18.644 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-20T12:51:18.628+0000 7f784c485700 1 -- 172.21.15.64:0/2757148432 wait complete. 2024-02-20T12:51:18.644 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-20T12:51:18.628+0000 7f784c485700 1 Processor -- start 2024-02-20T12:51:18.645 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-20T12:51:18.628+0000 7f784c485700 1 -- start start 2024-02-20T12:51:18.645 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-20T12:51:18.628+0000 7f784c485700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f78440f6590 0x7f78441067c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:51:18.645 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-20T12:51:18.628+0000 7f784c485700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7844106cb0 con 0x7f78440f6590 2024-02-20T12:51:18.645 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-20T12:51:18.628+0000 7f784a221700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f78440f6590 0x7f78441067c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:51:18.645 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-20T12:51:18.628+0000 7f784a221700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f78440f6590 0x7f78441067c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:38016/0 (socket says 172.21.15.64:38016) 2024-02-20T12:51:18.645 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-20T12:51:18.628+0000 7f784a221700 1 -- 172.21.15.64:0/3747009876 learned_addr learned my addr 172.21.15.64:0/3747009876 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-20T12:51:18.645 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-20T12:51:18.629+0000 7f784a221700 1 -- 172.21.15.64:0/3747009876 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7840009d70 con 0x7f78440f6590 2024-02-20T12:51:18.645 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-20T12:51:18.629+0000 7f784a221700 1 --2- 172.21.15.64:0/3747009876 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f78440f6590 0x7f78441067c0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 rx=0x7f7840004170 tx=0x7f7840009ad0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:51:18.645 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-20T12:51:18.629+0000 7f7836ffd700 1 -- 172.21.15.64:0/3747009876 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f784000e040 con 0x7f78440f6590 2024-02-20T12:51:18.645 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-20T12:51:18.629+0000 7f7836ffd700 1 -- 172.21.15.64:0/3747009876 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(25 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f7840008ef0 con 0x7f78440f6590 2024-02-20T12:51:18.645 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-20T12:51:18.629+0000 7f784c485700 1 -- 172.21.15.64:0/3747009876 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7844106eb0 con 0x7f78440f6590 2024-02-20T12:51:18.645 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-20T12:51:18.629+0000 7f7836ffd700 1 -- 172.21.15.64:0/3747009876 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7840014c90 con 0x7f78440f6590 2024-02-20T12:51:18.645 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-20T12:51:18.629+0000 7f784c485700 1 -- 172.21.15.64:0/3747009876 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7844107340 con 0x7f78440f6590 2024-02-20T12:51:18.645 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-20T12:51:18.630+0000 7f7836ffd700 1 -- 172.21.15.64:0/3747009876 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 37151+0+0 (secure 0 0 0) 0x7f78400283f0 con 0x7f78440f6590 2024-02-20T12:51:18.645 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-20T12:51:18.630+0000 7f7836ffd700 1 --2- 172.21.15.64:0/3747009876 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f78300310e0 0x7f78300335a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:51:18.646 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-20T12:51:18.630+0000 7f7836ffd700 1 -- 172.21.15.64:0/3747009876 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1065+0+0 (secure 0 0 0) 0x7f78400421f0 con 0x7f78440f6590 2024-02-20T12:51:18.646 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-20T12:51:18.630+0000 7f784c485700 1 -- 172.21.15.64:0/3747009876 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7844045bb0 con 0x7f78440f6590 2024-02-20T12:51:18.646 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-20T12:51:18.630+0000 7f7849a20700 1 --2- 172.21.15.64:0/3747009876 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f78300310e0 0x7f78300335a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:51:18.646 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-20T12:51:18.630+0000 7f7849a20700 1 --2- 172.21.15.64:0/3747009876 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f78300310e0 0x7f78300335a0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 rx=0x7f78380096a0 tx=0x7f7838006b40).ready entity=mgr.14156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:51:18.646 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-20T12:51:18.634+0000 7f7836ffd700 1 -- 172.21.15.64:0/3747009876 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+153237 (secure 0 0 0) 0x7f78400268f0 con 0x7f78440f6590 2024-02-20T12:51:18.800 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-20T12:51:18.799+0000 7f784c485700 1 -- 172.21.15.64:0/3747009876 --> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f78440f2ed0 con 0x7f78300310e0 2024-02-20T12:51:18.801 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-20T12:51:18.800+0000 7f7836ffd700 1 -- 172.21.15.64:0/3747009876 <== mgr.14156 v2:172.21.15.64:6800/1408391390 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+159 (secure 0 0 0) 0x7f78440f2ed0 con 0x7f78300310e0 2024-02-20T12:51:18.801 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-02-20T12:51:18.801 INFO:teuthology.orchestra.run.smithi064.stdout:[{"addr": "172.21.15.64", "hostname": "smithi064", "labels": [], "status": ""}, {"addr": "172.21.15.71", "hostname": "smithi071", "labels": [], "status": ""}] 2024-02-20T12:51:18.803 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-20T12:51:18.802+0000 7f784c485700 1 -- 172.21.15.64:0/3747009876 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f78300310e0 msgr2=0x7f78300335a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:51:18.803 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-20T12:51:18.802+0000 7f784c485700 1 --2- 172.21.15.64:0/3747009876 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f78300310e0 0x7f78300335a0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 rx=0x7f78380096a0 tx=0x7f7838006b40).stop 2024-02-20T12:51:18.803 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-20T12:51:18.802+0000 7f784c485700 1 -- 172.21.15.64:0/3747009876 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f78440f6590 msgr2=0x7f78441067c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:51:18.803 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-20T12:51:18.802+0000 7f784c485700 1 --2- 172.21.15.64:0/3747009876 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f78440f6590 0x7f78441067c0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 rx=0x7f7840004170 tx=0x7f7840009ad0).stop 2024-02-20T12:51:18.803 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-20T12:51:18.802+0000 7f784c485700 1 -- 172.21.15.64:0/3747009876 shutdown_connections 2024-02-20T12:51:18.803 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-20T12:51:18.802+0000 7f784c485700 1 --2- 172.21.15.64:0/3747009876 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f78300310e0 0x7f78300335a0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:51:18.803 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-20T12:51:18.802+0000 7f784c485700 1 --2- 172.21.15.64:0/3747009876 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f78440f6590 0x7f78441067c0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:51:18.803 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-20T12:51:18.802+0000 7f784c485700 1 -- 172.21.15.64:0/3747009876 >> 172.21.15.64:0/3747009876 conn(0x7f78440f1cd0 msgr2=0x7f78440f27c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:51:18.803 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-20T12:51:18.802+0000 7f784c485700 1 -- 172.21.15.64:0/3747009876 shutdown_connections 2024-02-20T12:51:18.804 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-20T12:51:18.802+0000 7f784c485700 1 -- 172.21.15.64:0/3747009876 wait complete. 2024-02-20T12:51:19.536 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:51:19 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:51:17.703853+0000 mgr.smithi064.vriyor (mgr.14156) 11 : cluster [DBG] pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:51:19.558 INFO:tasks.cephadm:Setting crush tunables to default 2024-02-20T12:51:19.558 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/daemon-base:latest-pacific shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 680ee126-cfee-11ee-95bb-87774f69a715 -- ceph osd crush tunables default 2024-02-20T12:51:20.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:51:20 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:51:18.801208+0000 mgr.smithi064.vriyor (mgr.14156) 12 : audit [DBG] from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-20T12:51:21.450 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:51:21 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:51:19.704093+0000 mgr.smithi064.vriyor (mgr.14156) 13 : cluster [DBG] pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:51:21.712 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-20T12:51:21.710+0000 7f4b96cf6700 1 -- 172.21.15.64:0/3761657692 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4b900f52d0 msgr2=0x7f4b900f56f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:51:21.712 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-20T12:51:21.710+0000 7f4b96cf6700 1 --2- 172.21.15.64:0/3761657692 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4b900f52d0 0x7f4b900f56f0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 rx=0x7f4b80009fb0 tx=0x7f4b80009b30).stop 2024-02-20T12:51:21.712 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-20T12:51:21.710+0000 7f4b96cf6700 1 -- 172.21.15.64:0/3761657692 shutdown_connections 2024-02-20T12:51:21.712 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-20T12:51:21.710+0000 7f4b96cf6700 1 --2- 172.21.15.64:0/3761657692 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4b900f52d0 0x7f4b900f56f0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:51:21.712 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-20T12:51:21.710+0000 7f4b96cf6700 1 -- 172.21.15.64:0/3761657692 >> 172.21.15.64:0/3761657692 conn(0x7f4b900f0a30 msgr2=0x7f4b900f2e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:51:21.712 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-20T12:51:21.710+0000 7f4b96cf6700 1 -- 172.21.15.64:0/3761657692 shutdown_connections 2024-02-20T12:51:21.712 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-20T12:51:21.710+0000 7f4b96cf6700 1 -- 172.21.15.64:0/3761657692 wait complete. 2024-02-20T12:51:21.713 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-20T12:51:21.711+0000 7f4b96cf6700 1 Processor -- start 2024-02-20T12:51:21.713 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-20T12:51:21.711+0000 7f4b96cf6700 1 -- start start 2024-02-20T12:51:21.713 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-20T12:51:21.711+0000 7f4b96cf6700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4b900f52d0 0x7f4b90109330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:51:21.713 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-20T12:51:21.711+0000 7f4b96cf6700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b90060300 con 0x7f4b900f52d0 2024-02-20T12:51:21.713 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-20T12:51:21.711+0000 7f4b95cf4700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4b900f52d0 0x7f4b90109330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:51:21.713 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-20T12:51:21.711+0000 7f4b95cf4700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4b900f52d0 0x7f4b90109330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:50202/0 (socket says 172.21.15.64:50202) 2024-02-20T12:51:21.713 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-20T12:51:21.711+0000 7f4b95cf4700 1 -- 172.21.15.64:0/132817158 learned_addr learned my addr 172.21.15.64:0/132817158 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-02-20T12:51:21.713 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-20T12:51:21.711+0000 7f4b95cf4700 1 -- 172.21.15.64:0/132817158 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b80005040 con 0x7f4b900f52d0 2024-02-20T12:51:21.713 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-20T12:51:21.711+0000 7f4b95cf4700 1 --2- 172.21.15.64:0/132817158 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4b900f52d0 0x7f4b90109330 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 rx=0x7f4b800089f0 tx=0x7f4b8000b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:51:21.714 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-20T12:51:21.711+0000 7f4b867fc700 1 -- 172.21.15.64:0/132817158 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4b80009320 con 0x7f4b900f52d0 2024-02-20T12:51:21.714 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-20T12:51:21.712+0000 7f4b867fc700 1 -- 172.21.15.64:0/132817158 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(25 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f4b80013ec0 con 0x7f4b900f52d0 2024-02-20T12:51:21.714 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-20T12:51:21.712+0000 7f4b96cf6700 1 -- 172.21.15.64:0/132817158 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b90060500 con 0x7f4b900f52d0 2024-02-20T12:51:21.715 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-20T12:51:21.712+0000 7f4b867fc700 1 -- 172.21.15.64:0/132817158 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4b80014d30 con 0x7f4b900f52d0 2024-02-20T12:51:21.715 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-20T12:51:21.712+0000 7f4b96cf6700 1 -- 172.21.15.64:0/132817158 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b901099e0 con 0x7f4b900f52d0 2024-02-20T12:51:21.715 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-20T12:51:21.712+0000 7f4b867fc700 1 -- 172.21.15.64:0/132817158 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 37151+0+0 (secure 0 0 0) 0x7f4b800095f0 con 0x7f4b900f52d0 2024-02-20T12:51:21.715 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-20T12:51:21.713+0000 7f4b96cf6700 1 -- 172.21.15.64:0/132817158 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b900591e0 con 0x7f4b900f52d0 2024-02-20T12:51:21.715 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-20T12:51:21.713+0000 7f4b867fc700 1 --2- 172.21.15.64:0/132817158 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f4b7c0310e0 0x7f4b7c0335a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:51:21.715 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-20T12:51:21.713+0000 7f4b867fc700 1 -- 172.21.15.64:0/132817158 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1065+0+0 (secure 0 0 0) 0x7f4b800424a0 con 0x7f4b900f52d0 2024-02-20T12:51:21.715 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-20T12:51:21.713+0000 7f4b954f3700 1 --2- 172.21.15.64:0/132817158 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f4b7c0310e0 0x7f4b7c0335a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:51:21.715 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-20T12:51:21.713+0000 7f4b954f3700 1 --2- 172.21.15.64:0/132817158 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f4b7c0310e0 0x7f4b7c0335a0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 rx=0x7f4b8c0096a0 tx=0x7f4b8c006b40).ready entity=mgr.14156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:51:21.718 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-20T12:51:21.717+0000 7f4b867fc700 1 -- 172.21.15.64:0/132817158 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+153237 (secure 0 0 0) 0x7f4b8001d980 con 0x7f4b900f52d0 2024-02-20T12:51:21.883 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-20T12:51:21.883+0000 7f4b96cf6700 1 -- 172.21.15.64:0/132817158 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f4b90045bb0 con 0x7f4b900f52d0 2024-02-20T12:51:22.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-20T12:51:22.344+0000 7f4b867fc700 1 -- 172.21.15.64:0/132817158 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) v1 ==== 124+0+0 (secure 0 0 0) 0x7f4b8001d380 con 0x7f4b900f52d0 2024-02-20T12:51:22.346 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-20T12:51:22.345+0000 7f4b96cf6700 1 -- 172.21.15.64:0/132817158 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f4b7c0310e0 msgr2=0x7f4b7c0335a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:51:22.347 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-20T12:51:22.345+0000 7f4b96cf6700 1 --2- 172.21.15.64:0/132817158 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f4b7c0310e0 0x7f4b7c0335a0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 rx=0x7f4b8c0096a0 tx=0x7f4b8c006b40).stop 2024-02-20T12:51:22.347 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-20T12:51:22.346+0000 7f4b96cf6700 1 -- 172.21.15.64:0/132817158 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4b900f52d0 msgr2=0x7f4b90109330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:51:22.347 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-20T12:51:22.346+0000 7f4b96cf6700 1 --2- 172.21.15.64:0/132817158 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4b900f52d0 0x7f4b90109330 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 rx=0x7f4b800089f0 tx=0x7f4b8000b040).stop 2024-02-20T12:51:22.347 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-20T12:51:22.346+0000 7f4b96cf6700 1 -- 172.21.15.64:0/132817158 shutdown_connections 2024-02-20T12:51:22.347 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-20T12:51:22.346+0000 7f4b96cf6700 1 --2- 172.21.15.64:0/132817158 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f4b7c0310e0 0x7f4b7c0335a0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:51:22.347 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-20T12:51:22.346+0000 7f4b96cf6700 1 --2- 172.21.15.64:0/132817158 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4b900f52d0 0x7f4b90109330 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:51:22.347 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-20T12:51:22.346+0000 7f4b96cf6700 1 -- 172.21.15.64:0/132817158 >> 172.21.15.64:0/132817158 conn(0x7f4b900f0a30 msgr2=0x7f4b900f1550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:51:22.347 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-20T12:51:22.346+0000 7f4b96cf6700 1 -- 172.21.15.64:0/132817158 shutdown_connections 2024-02-20T12:51:22.347 INFO:teuthology.orchestra.run.smithi064.stderr:2024-02-20T12:51:22.346+0000 7f4b96cf6700 1 -- 172.21.15.64:0/132817158 wait complete. 2024-02-20T12:51:22.347 INFO:teuthology.orchestra.run.smithi064.stderr:adjusted tunables profile to default 2024-02-20T12:51:22.692 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:51:22 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:51:21.884272+0000 mon.smithi064 (mon.0) 102 : audit [INF] from='client.? 172.21.15.64:0/132817158' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-02-20T12:51:23.034 INFO:tasks.cephadm:Adding mon.smithi064 on smithi064 2024-02-20T12:51:23.034 INFO:tasks.cephadm:Adding mon.smithi071 on smithi071 2024-02-20T12:51:23.034 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/daemon-base:latest-pacific shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 680ee126-cfee-11ee-95bb-87774f69a715 -- ceph orch apply mon '2;smithi064:172.21.15.64=smithi064;smithi071:172.21.15.71=smithi071' 2024-02-20T12:51:23.812 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:51:23 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:51:21.704300+0000 mgr.smithi064.vriyor (mgr.14156) 14 : cluster [DBG] pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:51:23.812 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:51:23 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:51:22.344913+0000 mon.smithi064 (mon.0) 103 : audit [INF] from='client.? 172.21.15.64:0/132817158' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-02-20T12:51:23.812 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:51:23 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:51:22.344962+0000 mon.smithi064 (mon.0) 104 : cluster [DBG] osdmap e4: 0 total, 0 up, 0 in 2024-02-20T12:51:24.941 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:24.939+0000 7f6379729700 1 -- 172.21.15.71:0/4033348180 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f63740f52d0 msgr2=0x7f63740f56f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:51:24.941 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:24.939+0000 7f6379729700 1 --2- 172.21.15.71:0/4033348180 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f63740f52d0 0x7f63740f56f0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 rx=0x7f635c009fb0 tx=0x7f635c009b30).stop 2024-02-20T12:51:24.941 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:24.939+0000 7f6379729700 1 -- 172.21.15.71:0/4033348180 shutdown_connections 2024-02-20T12:51:24.941 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:24.939+0000 7f6379729700 1 --2- 172.21.15.71:0/4033348180 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f63740f52d0 0x7f63740f56f0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:51:24.941 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:24.939+0000 7f6379729700 1 -- 172.21.15.71:0/4033348180 >> 172.21.15.71:0/4033348180 conn(0x7f63740f0a30 msgr2=0x7f63740f2e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:51:24.941 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:24.939+0000 7f6379729700 1 -- 172.21.15.71:0/4033348180 shutdown_connections 2024-02-20T12:51:24.942 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:24.939+0000 7f6379729700 1 -- 172.21.15.71:0/4033348180 wait complete. 2024-02-20T12:51:24.942 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:24.939+0000 7f6379729700 1 Processor -- start 2024-02-20T12:51:24.942 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:24.940+0000 7f6379729700 1 -- start start 2024-02-20T12:51:24.942 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:24.940+0000 7f6379729700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f63740f52d0 0x7f63740fd2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:51:24.942 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:24.940+0000 7f6379729700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63740fb730 con 0x7f63740f52d0 2024-02-20T12:51:24.942 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:24.940+0000 7f6373fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f63740f52d0 0x7f63740fd2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:51:24.942 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:24.940+0000 7f6373fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f63740f52d0 0x7f63740fd2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.71:49496/0 (socket says 172.21.15.71:49496) 2024-02-20T12:51:24.942 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:24.940+0000 7f6373fff700 1 -- 172.21.15.71:0/1588216715 learned_addr learned my addr 172.21.15.71:0/1588216715 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-02-20T12:51:24.942 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:24.940+0000 7f6373fff700 1 -- 172.21.15.71:0/1588216715 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f635c005040 con 0x7f63740f52d0 2024-02-20T12:51:24.942 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:24.940+0000 7f6373fff700 1 --2- 172.21.15.71:0/1588216715 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f63740f52d0 0x7f63740fd2b0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 rx=0x7f635c0089f0 tx=0x7f635c00b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:51:24.942 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:24.941+0000 7f6370ff9700 1 -- 172.21.15.71:0/1588216715 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f635c009320 con 0x7f63740f52d0 2024-02-20T12:51:24.942 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:24.941+0000 7f6370ff9700 1 -- 172.21.15.71:0/1588216715 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(25 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f635c013ec0 con 0x7f63740f52d0 2024-02-20T12:51:24.942 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:24.941+0000 7f6370ff9700 1 -- 172.21.15.71:0/1588216715 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f635c014d30 con 0x7f63740f52d0 2024-02-20T12:51:24.942 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:24.941+0000 7f6379729700 1 -- 172.21.15.71:0/1588216715 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63740fb8d0 con 0x7f63740f52d0 2024-02-20T12:51:24.942 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:24.941+0000 7f6379729700 1 -- 172.21.15.71:0/1588216715 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63740fbd60 con 0x7f63740f52d0 2024-02-20T12:51:24.943 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:24.941+0000 7f6370ff9700 1 -- 172.21.15.71:0/1588216715 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 37151+0+0 (secure 0 0 0) 0x7f635c0095f0 con 0x7f63740f52d0 2024-02-20T12:51:24.943 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:24.941+0000 7f6379729700 1 -- 172.21.15.71:0/1588216715 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63740591e0 con 0x7f63740f52d0 2024-02-20T12:51:24.944 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:24.942+0000 7f6370ff9700 1 --2- 172.21.15.71:0/1588216715 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f6360031090 0x7f6360033550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:51:24.944 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:24.942+0000 7f6370ff9700 1 -- 172.21.15.71:0/1588216715 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1065+0+0 (secure 0 0 0) 0x7f635c0424a0 con 0x7f63740f52d0 2024-02-20T12:51:24.944 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:24.942+0000 7f63737fe700 1 --2- 172.21.15.71:0/1588216715 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f6360031090 0x7f6360033550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:51:24.944 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:24.942+0000 7f63737fe700 1 --2- 172.21.15.71:0/1588216715 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f6360031090 0x7f6360033550 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 rx=0x7f63640096a0 tx=0x7f6364006b40).ready entity=mgr.14156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:51:24.947 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:24.946+0000 7f6370ff9700 1 -- 172.21.15.71:0/1588216715 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+153237 (secure 0 0 0) 0x7f635c01d7f0 con 0x7f63740f52d0 2024-02-20T12:51:25.104 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:25.103+0000 7f6379729700 1 -- 172.21.15.71:0/1588216715 --> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "2;smithi064:172.21.15.64=smithi064;smithi071:172.21.15.71=smithi071", "target": ["mon-mgr", ""]}) v1 -- 0x7f63740fc660 con 0x7f6360031090 2024-02-20T12:51:25.109 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:25.109+0000 7f6370ff9700 1 -- 172.21.15.71:0/1588216715 <== mgr.14156 v2:172.21.15.64:6800/1408391390 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f63740fc660 con 0x7f6360031090 2024-02-20T12:51:25.109 INFO:teuthology.orchestra.run.smithi071.stdout:Scheduled mon update... 2024-02-20T12:51:25.111 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:25.110+0000 7f6379729700 1 -- 172.21.15.71:0/1588216715 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f6360031090 msgr2=0x7f6360033550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:51:25.111 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:25.110+0000 7f6379729700 1 --2- 172.21.15.71:0/1588216715 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f6360031090 0x7f6360033550 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 rx=0x7f63640096a0 tx=0x7f6364006b40).stop 2024-02-20T12:51:25.111 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:25.110+0000 7f6379729700 1 -- 172.21.15.71:0/1588216715 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f63740f52d0 msgr2=0x7f63740fd2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:51:25.111 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:25.110+0000 7f6379729700 1 --2- 172.21.15.71:0/1588216715 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f63740f52d0 0x7f63740fd2b0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 rx=0x7f635c0089f0 tx=0x7f635c00b040).stop 2024-02-20T12:51:25.111 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:25.110+0000 7f6379729700 1 -- 172.21.15.71:0/1588216715 shutdown_connections 2024-02-20T12:51:25.111 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:25.110+0000 7f6379729700 1 --2- 172.21.15.71:0/1588216715 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f6360031090 0x7f6360033550 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:51:25.111 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:25.110+0000 7f6379729700 1 --2- 172.21.15.71:0/1588216715 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f63740f52d0 0x7f63740fd2b0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:51:25.111 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:25.110+0000 7f6379729700 1 -- 172.21.15.71:0/1588216715 >> 172.21.15.71:0/1588216715 conn(0x7f63740f0a30 msgr2=0x7f637405ff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:51:25.111 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:25.110+0000 7f6379729700 1 -- 172.21.15.71:0/1588216715 shutdown_connections 2024-02-20T12:51:25.111 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:25.110+0000 7f6379729700 1 -- 172.21.15.71:0/1588216715 wait complete. 2024-02-20T12:51:25.721 DEBUG:teuthology.orchestra.run.smithi071:mon.smithi071> sudo journalctl -f -n 0 -u ceph-680ee126-cfee-11ee-95bb-87774f69a715@mon.smithi071.service 2024-02-20T12:51:25.724 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-20T12:51:25.724 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/daemon-base:latest-pacific shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 680ee126-cfee-11ee-95bb-87774f69a715 -- ceph mon dump -f json 2024-02-20T12:51:25.753 INFO:journalctl@ceph.mon.smithi071.smithi071.stdout:-- Logs begin at Tue 2024-02-20 12:38:38 UTC. -- 2024-02-20T12:51:25.812 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:51:25 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:51:23.704545+0000 mgr.smithi064.vriyor (mgr.14156) 15 : cluster [DBG] pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:51:25.812 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:51:25 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:51:25.109022+0000 mon.smithi064 (mon.0) 105 : audit [INF] from='mgr.14156 172.21.15.64:0/3284974785' entity='mgr.smithi064.vriyor' 2024-02-20T12:51:26.812 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:51:26 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:51:25.104827+0000 mgr.smithi064.vriyor (mgr.14156) 16 : audit [DBG] from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "2;smithi064:172.21.15.64=smithi064;smithi071:172.21.15.71=smithi071", "target": ["mon-mgr", ""]}]: dispatch 2024-02-20T12:51:26.812 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:51:26 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cephadm 2024-02-20T12:51:25.106324+0000 mgr.smithi064.vriyor (mgr.14156) 17 : cephadm [INF] Saving service mon spec with placement smithi064:172.21.15.64=smithi064;smithi071:172.21.15.71=smithi071;count:2 2024-02-20T12:51:27.408 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:27.406+0000 7f326a546700 1 -- 172.21.15.71:0/2271638540 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f32640f52d0 msgr2=0x7f32640f56f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:51:27.408 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:27.406+0000 7f326a546700 1 --2- 172.21.15.71:0/2271638540 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f32640f52d0 0x7f32640f56f0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 rx=0x7f3260009fb0 tx=0x7f3260009b30).stop 2024-02-20T12:51:27.409 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:27.406+0000 7f326a546700 1 -- 172.21.15.71:0/2271638540 shutdown_connections 2024-02-20T12:51:27.409 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:27.406+0000 7f326a546700 1 --2- 172.21.15.71:0/2271638540 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f32640f52d0 0x7f32640f56f0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:51:27.409 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:27.406+0000 7f326a546700 1 -- 172.21.15.71:0/2271638540 >> 172.21.15.71:0/2271638540 conn(0x7f32640f0a30 msgr2=0x7f32640f2e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:51:27.409 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:27.406+0000 7f326a546700 1 -- 172.21.15.71:0/2271638540 shutdown_connections 2024-02-20T12:51:27.409 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:27.406+0000 7f326a546700 1 -- 172.21.15.71:0/2271638540 wait complete. 2024-02-20T12:51:27.409 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:27.407+0000 7f326a546700 1 Processor -- start 2024-02-20T12:51:27.409 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:27.407+0000 7f326a546700 1 -- start start 2024-02-20T12:51:27.409 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:27.407+0000 7f326a546700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f32640f52d0 0x7f3264109310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:51:27.409 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:27.407+0000 7f326a546700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3264060320 con 0x7f32640f52d0 2024-02-20T12:51:27.409 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:27.407+0000 7f3269544700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f32640f52d0 0x7f3264109310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:51:27.409 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:27.407+0000 7f3269544700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f32640f52d0 0x7f3264109310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.71:52580/0 (socket says 172.21.15.71:52580) 2024-02-20T12:51:27.409 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:27.407+0000 7f3269544700 1 -- 172.21.15.71:0/3041146585 learned_addr learned my addr 172.21.15.71:0/3041146585 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-02-20T12:51:27.410 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:27.407+0000 7f3269544700 1 -- 172.21.15.71:0/3041146585 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3260005040 con 0x7f32640f52d0 2024-02-20T12:51:27.410 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:27.407+0000 7f3269544700 1 --2- 172.21.15.71:0/3041146585 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f32640f52d0 0x7f3264109310 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 rx=0x7f3260000c00 tx=0x7f326000b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:51:27.410 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:27.408+0000 7f3259ffb700 1 -- 172.21.15.71:0/3041146585 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f32600095f0 con 0x7f32640f52d0 2024-02-20T12:51:27.410 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:27.408+0000 7f326a546700 1 -- 172.21.15.71:0/3041146585 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3264060520 con 0x7f32640f52d0 2024-02-20T12:51:27.410 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:27.408+0000 7f3259ffb700 1 -- 172.21.15.71:0/3041146585 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(25 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f3260007e50 con 0x7f32640f52d0 2024-02-20T12:51:27.410 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:27.408+0000 7f3259ffb700 1 -- 172.21.15.71:0/3041146585 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3260014e70 con 0x7f32640f52d0 2024-02-20T12:51:27.410 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:27.408+0000 7f326a546700 1 -- 172.21.15.71:0/3041146585 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32641099c0 con 0x7f32640f52d0 2024-02-20T12:51:27.411 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:27.409+0000 7f3259ffb700 1 -- 172.21.15.71:0/3041146585 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 37151+0+0 (secure 0 0 0) 0x7f3260025070 con 0x7f32640f52d0 2024-02-20T12:51:27.411 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:27.409+0000 7f326a546700 1 -- 172.21.15.71:0/3041146585 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3264045bb0 con 0x7f32640f52d0 2024-02-20T12:51:27.411 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:27.409+0000 7f3259ffb700 1 --2- 172.21.15.71:0/3041146585 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f3250031090 0x7f3250033550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:51:27.411 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:27.409+0000 7f3259ffb700 1 -- 172.21.15.71:0/3041146585 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1065+0+0 (secure 0 0 0) 0x7f3260041d20 con 0x7f32640f52d0 2024-02-20T12:51:27.411 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:27.409+0000 7f3268d43700 1 --2- 172.21.15.71:0/3041146585 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f3250031090 0x7f3250033550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:51:27.411 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:27.410+0000 7f3268d43700 1 --2- 172.21.15.71:0/3041146585 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f3250031090 0x7f3250033550 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 rx=0x7f32540096a0 tx=0x7f3254006b40).ready entity=mgr.14156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:51:27.414 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:27.414+0000 7f3259ffb700 1 -- 172.21.15.71:0/3041146585 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+153237 (secure 0 0 0) 0x7f3260014420 con 0x7f32640f52d0 2024-02-20T12:51:27.619 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:27.619+0000 7f326a546700 1 -- 172.21.15.71:0/3041146585 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f326410a1a0 con 0x7f32640f52d0 2024-02-20T12:51:27.620 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:27.619+0000 7f3259ffb700 1 -- 172.21.15.71:0/3041146585 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7f326000f300 con 0x7f32640f52d0 2024-02-20T12:51:27.620 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-20T12:51:27.620 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":1,"fsid":"680ee126-cfee-11ee-95bb-87774f69a715","modified":"2024-02-20T12:49:44.699059Z","created":"2024-02-20T12:49:44.699059Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi064","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:3300","nonce":0},{"type":"v1","addr":"172.21.15.64:6789","nonce":0}]},"addr":"172.21.15.64:6789/0","public_addr":"172.21.15.64:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-20T12:51:27.622 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:27.621+0000 7f326a546700 1 -- 172.21.15.71:0/3041146585 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f3250031090 msgr2=0x7f3250033550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:51:27.622 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:27.621+0000 7f326a546700 1 --2- 172.21.15.71:0/3041146585 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f3250031090 0x7f3250033550 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 rx=0x7f32540096a0 tx=0x7f3254006b40).stop 2024-02-20T12:51:27.622 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:27.621+0000 7f326a546700 1 -- 172.21.15.71:0/3041146585 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f32640f52d0 msgr2=0x7f3264109310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:51:27.622 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:27.621+0000 7f326a546700 1 --2- 172.21.15.71:0/3041146585 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f32640f52d0 0x7f3264109310 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 rx=0x7f3260000c00 tx=0x7f326000b040).stop 2024-02-20T12:51:27.622 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:27.621+0000 7f326a546700 1 -- 172.21.15.71:0/3041146585 shutdown_connections 2024-02-20T12:51:27.622 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:27.621+0000 7f326a546700 1 --2- 172.21.15.71:0/3041146585 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f3250031090 0x7f3250033550 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:51:27.622 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:27.621+0000 7f326a546700 1 --2- 172.21.15.71:0/3041146585 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f32640f52d0 0x7f3264109310 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:51:27.622 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:27.621+0000 7f326a546700 1 -- 172.21.15.71:0/3041146585 >> 172.21.15.71:0/3041146585 conn(0x7f32640f0a30 msgr2=0x7f32640f1550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:51:27.622 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:27.621+0000 7f326a546700 1 -- 172.21.15.71:0/3041146585 shutdown_connections 2024-02-20T12:51:27.622 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:27.621+0000 7f326a546700 1 -- 172.21.15.71:0/3041146585 wait complete. 2024-02-20T12:51:27.623 INFO:teuthology.orchestra.run.smithi071.stderr:dumped monmap epoch 1 2024-02-20T12:51:27.812 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:51:27 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:51:25.704903+0000 mgr.smithi064.vriyor (mgr.14156) 18 : cluster [DBG] pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:51:28.812 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:51:28 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:51:27.620260+0000 mon.smithi064 (mon.0) 106 : audit [DBG] from='client.? 172.21.15.71:0/3041146585' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-20T12:51:28.812 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:51:28 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:51:27.705272+0000 mgr.smithi064.vriyor (mgr.14156) 19 : cluster [DBG] pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:51:29.300 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-20T12:51:29.300 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/daemon-base:latest-pacific shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 680ee126-cfee-11ee-95bb-87774f69a715 -- ceph mon dump -f json 2024-02-20T12:51:31.092 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:31.081+0000 7f2489257700 1 -- 172.21.15.71:0/1535217204 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f24840f52e0 msgr2=0x7f24840f5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:51:31.092 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:31.081+0000 7f2489257700 1 --2- 172.21.15.71:0/1535217204 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f24840f52e0 0x7f24840f5700 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 rx=0x7f2474009fb0 tx=0x7f2474009b30).stop 2024-02-20T12:51:31.092 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:31.081+0000 7f2489257700 1 -- 172.21.15.71:0/1535217204 shutdown_connections 2024-02-20T12:51:31.092 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:31.081+0000 7f2489257700 1 --2- 172.21.15.71:0/1535217204 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f24840f52e0 0x7f24840f5700 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:51:31.092 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:31.081+0000 7f2489257700 1 -- 172.21.15.71:0/1535217204 >> 172.21.15.71:0/1535217204 conn(0x7f24840f0a40 msgr2=0x7f24840f2e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:51:31.092 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:31.081+0000 7f2489257700 1 -- 172.21.15.71:0/1535217204 shutdown_connections 2024-02-20T12:51:31.092 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:31.081+0000 7f2489257700 1 -- 172.21.15.71:0/1535217204 wait complete. 2024-02-20T12:51:31.092 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:31.082+0000 7f2489257700 1 Processor -- start 2024-02-20T12:51:31.092 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:31.082+0000 7f2489257700 1 -- start start 2024-02-20T12:51:31.092 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:31.082+0000 7f2489257700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f24840f52e0 0x7f2484105500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:51:31.093 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:31.082+0000 7f2489257700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24841059f0 con 0x7f24840f52e0 2024-02-20T12:51:31.093 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:31.083+0000 7f2483fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f24840f52e0 0x7f2484105500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:51:31.093 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:31.083+0000 7f2483fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f24840f52e0 0x7f2484105500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.71:52594/0 (socket says 172.21.15.71:52594) 2024-02-20T12:51:31.093 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:31.083+0000 7f2483fff700 1 -- 172.21.15.71:0/3819387283 learned_addr learned my addr 172.21.15.71:0/3819387283 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-02-20T12:51:31.093 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:31.083+0000 7f2483fff700 1 -- 172.21.15.71:0/3819387283 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2474005040 con 0x7f24840f52e0 2024-02-20T12:51:31.093 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:31.083+0000 7f2483fff700 1 --2- 172.21.15.71:0/3819387283 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f24840f52e0 0x7f2484105500 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 rx=0x7f24740089f0 tx=0x7f247400b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:51:31.093 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:31.083+0000 7f2480ff9700 1 -- 172.21.15.71:0/3819387283 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2474009320 con 0x7f24840f52e0 2024-02-20T12:51:31.093 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:31.083+0000 7f2489257700 1 -- 172.21.15.71:0/3819387283 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2484105bf0 con 0x7f24840f52e0 2024-02-20T12:51:31.093 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:31.083+0000 7f2480ff9700 1 -- 172.21.15.71:0/3819387283 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(25 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f2474013ec0 con 0x7f24840f52e0 2024-02-20T12:51:31.093 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:31.083+0000 7f2480ff9700 1 -- 172.21.15.71:0/3819387283 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2474014d30 con 0x7f24840f52e0 2024-02-20T12:51:31.093 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:31.083+0000 7f2489257700 1 -- 172.21.15.71:0/3819387283 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2484106080 con 0x7f24840f52e0 2024-02-20T12:51:31.093 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:31.084+0000 7f2480ff9700 1 -- 172.21.15.71:0/3819387283 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 37151+0+0 (secure 0 0 0) 0x7f24740095f0 con 0x7f24840f52e0 2024-02-20T12:51:31.093 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:31.084+0000 7f2489257700 1 -- 172.21.15.71:0/3819387283 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2484045bb0 con 0x7f24840f52e0 2024-02-20T12:51:31.093 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:31.084+0000 7f2480ff9700 1 --2- 172.21.15.71:0/3819387283 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f246c031090 0x7f246c033550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:51:31.094 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:31.084+0000 7f2480ff9700 1 -- 172.21.15.71:0/3819387283 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1065+0+0 (secure 0 0 0) 0x7f24740424e0 con 0x7f24840f52e0 2024-02-20T12:51:31.094 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:31.085+0000 7f24837fe700 1 --2- 172.21.15.71:0/3819387283 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f246c031090 0x7f246c033550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:51:31.094 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:31.085+0000 7f24837fe700 1 --2- 172.21.15.71:0/3819387283 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f246c031090 0x7f246c033550 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 rx=0x7f24780096a0 tx=0x7f2478006b40).ready entity=mgr.14156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:51:31.094 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:31.089+0000 7f2480ff9700 1 -- 172.21.15.71:0/3819387283 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+153237 (secure 0 0 0) 0x7f247401d7f0 con 0x7f24840f52e0 2024-02-20T12:51:31.296 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:31.296+0000 7f2489257700 1 -- 172.21.15.71:0/3819387283 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f2484106ed0 con 0x7f24840f52e0 2024-02-20T12:51:31.297 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:31.296+0000 7f2480ff9700 1 -- 172.21.15.71:0/3819387283 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7f247400fe40 con 0x7f24840f52e0 2024-02-20T12:51:31.297 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-20T12:51:31.297 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":1,"fsid":"680ee126-cfee-11ee-95bb-87774f69a715","modified":"2024-02-20T12:49:44.699059Z","created":"2024-02-20T12:49:44.699059Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi064","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:3300","nonce":0},{"type":"v1","addr":"172.21.15.64:6789","nonce":0}]},"addr":"172.21.15.64:6789/0","public_addr":"172.21.15.64:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-20T12:51:31.298 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:31.298+0000 7f2489257700 1 -- 172.21.15.71:0/3819387283 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f246c031090 msgr2=0x7f246c033550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:51:31.298 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:31.298+0000 7f2489257700 1 --2- 172.21.15.71:0/3819387283 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f246c031090 0x7f246c033550 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 rx=0x7f24780096a0 tx=0x7f2478006b40).stop 2024-02-20T12:51:31.298 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:31.298+0000 7f2489257700 1 -- 172.21.15.71:0/3819387283 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f24840f52e0 msgr2=0x7f2484105500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:51:31.299 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:31.298+0000 7f2489257700 1 --2- 172.21.15.71:0/3819387283 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f24840f52e0 0x7f2484105500 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 rx=0x7f24740089f0 tx=0x7f247400b040).stop 2024-02-20T12:51:31.299 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:31.298+0000 7f2489257700 1 -- 172.21.15.71:0/3819387283 shutdown_connections 2024-02-20T12:51:31.299 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:31.298+0000 7f2489257700 1 --2- 172.21.15.71:0/3819387283 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f246c031090 0x7f246c033550 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:51:31.299 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:31.298+0000 7f2489257700 1 --2- 172.21.15.71:0/3819387283 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f24840f52e0 0x7f2484105500 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:51:31.299 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:31.298+0000 7f2489257700 1 -- 172.21.15.71:0/3819387283 >> 172.21.15.71:0/3819387283 conn(0x7f24840f0a40 msgr2=0x7f24840f1540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:51:31.299 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:31.298+0000 7f2489257700 1 -- 172.21.15.71:0/3819387283 shutdown_connections 2024-02-20T12:51:31.299 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:31.298+0000 7f2489257700 1 -- 172.21.15.71:0/3819387283 wait complete. 2024-02-20T12:51:31.299 INFO:teuthology.orchestra.run.smithi071.stderr:dumped monmap epoch 1 2024-02-20T12:51:31.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:51:30 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:51:29.705642+0000 mgr.smithi064.vriyor (mgr.14156) 20 : cluster [DBG] pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:51:32.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:51:31 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:51:31.297209+0000 mon.smithi064 (mon.0) 107 : audit [DBG] from='client.? 172.21.15.71:0/3819387283' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-20T12:51:32.869 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-20T12:51:32.869 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/daemon-base:latest-pacific shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 680ee126-cfee-11ee-95bb-87774f69a715 -- ceph mon dump -f json 2024-02-20T12:51:33.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:51:32 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:51:31.706129+0000 mgr.smithi064.vriyor (mgr.14156) 21 : cluster [DBG] pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:51:34.698 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:34.697+0000 7f2dd5358700 1 -- 172.21.15.71:0/1222890687 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2dd00f52d0 msgr2=0x7f2dd00f56f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:51:34.699 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:34.697+0000 7f2dd5358700 1 --2- 172.21.15.71:0/1222890687 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2dd00f52d0 0x7f2dd00f56f0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 rx=0x7f2dc0009fb0 tx=0x7f2dc0009b30).stop 2024-02-20T12:51:34.699 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:34.698+0000 7f2dd5358700 1 -- 172.21.15.71:0/1222890687 shutdown_connections 2024-02-20T12:51:34.699 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:34.698+0000 7f2dd5358700 1 --2- 172.21.15.71:0/1222890687 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2dd00f52d0 0x7f2dd00f56f0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:51:34.699 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:34.698+0000 7f2dd5358700 1 -- 172.21.15.71:0/1222890687 >> 172.21.15.71:0/1222890687 conn(0x7f2dd00f0a30 msgr2=0x7f2dd00f2e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:51:34.699 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:34.698+0000 7f2dd5358700 1 -- 172.21.15.71:0/1222890687 shutdown_connections 2024-02-20T12:51:34.699 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:34.698+0000 7f2dd5358700 1 -- 172.21.15.71:0/1222890687 wait complete. 2024-02-20T12:51:34.699 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:34.698+0000 7f2dd5358700 1 Processor -- start 2024-02-20T12:51:34.699 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:34.698+0000 7f2dd5358700 1 -- start start 2024-02-20T12:51:34.701 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:34.698+0000 7f2dd5358700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2dd00f52d0 0x7f2dd00fd2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:51:34.701 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:34.698+0000 7f2dd5358700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dd00fb730 con 0x7f2dd00f52d0 2024-02-20T12:51:34.701 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:34.699+0000 7f2dcffff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2dd00f52d0 0x7f2dd00fd2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:51:34.701 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:34.699+0000 7f2dcffff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2dd00f52d0 0x7f2dd00fd2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.71:52614/0 (socket says 172.21.15.71:52614) 2024-02-20T12:51:34.701 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:34.699+0000 7f2dcffff700 1 -- 172.21.15.71:0/2680665168 learned_addr learned my addr 172.21.15.71:0/2680665168 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-02-20T12:51:34.701 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:34.699+0000 7f2dcffff700 1 -- 172.21.15.71:0/2680665168 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2dc0005040 con 0x7f2dd00f52d0 2024-02-20T12:51:34.701 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:34.699+0000 7f2dcffff700 1 --2- 172.21.15.71:0/2680665168 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2dd00f52d0 0x7f2dd00fd2b0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 rx=0x7f2dc00089f0 tx=0x7f2dc000b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:51:34.702 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:34.699+0000 7f2dccff9700 1 -- 172.21.15.71:0/2680665168 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2dc0009320 con 0x7f2dd00f52d0 2024-02-20T12:51:34.702 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:34.699+0000 7f2dccff9700 1 -- 172.21.15.71:0/2680665168 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(25 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f2dc0013ec0 con 0x7f2dd00f52d0 2024-02-20T12:51:34.702 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:34.699+0000 7f2dd5358700 1 -- 172.21.15.71:0/2680665168 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2dd00fb8d0 con 0x7f2dd00f52d0 2024-02-20T12:51:34.702 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:34.699+0000 7f2dccff9700 1 -- 172.21.15.71:0/2680665168 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2dc0014d30 con 0x7f2dd00f52d0 2024-02-20T12:51:34.702 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:34.699+0000 7f2dd5358700 1 -- 172.21.15.71:0/2680665168 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2dd00fbd60 con 0x7f2dd00f52d0 2024-02-20T12:51:34.702 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:34.700+0000 7f2dccff9700 1 -- 172.21.15.71:0/2680665168 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 37151+0+0 (secure 0 0 0) 0x7f2dc00263a0 con 0x7f2dd00f52d0 2024-02-20T12:51:34.702 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:34.700+0000 7f2dccff9700 1 --2- 172.21.15.71:0/2680665168 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f2db80310e0 0x7f2db80335a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:51:34.702 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:34.700+0000 7f2dccff9700 1 -- 172.21.15.71:0/2680665168 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1065+0+0 (secure 0 0 0) 0x7f2dc0042460 con 0x7f2dd00f52d0 2024-02-20T12:51:34.702 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:34.700+0000 7f2dcf7fe700 1 --2- 172.21.15.71:0/2680665168 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f2db80310e0 0x7f2db80335a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:51:34.702 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:34.700+0000 7f2dd5358700 1 -- 172.21.15.71:0/2680665168 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2dd00591e0 con 0x7f2dd00f52d0 2024-02-20T12:51:34.702 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:34.701+0000 7f2dcf7fe700 1 --2- 172.21.15.71:0/2680665168 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f2db80310e0 0x7f2db80335a0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 rx=0x7f2dc40096a0 tx=0x7f2dc4006b40).ready entity=mgr.14156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:51:34.706 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:34.705+0000 7f2dccff9700 1 -- 172.21.15.71:0/2680665168 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+153237 (secure 0 0 0) 0x7f2dc00266d0 con 0x7f2dd00f52d0 2024-02-20T12:51:34.912 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:34.912+0000 7f2dd5358700 1 -- 172.21.15.71:0/2680665168 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f2dd00fc3c0 con 0x7f2dd00f52d0 2024-02-20T12:51:34.913 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:34.913+0000 7f2dccff9700 1 -- 172.21.15.71:0/2680665168 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7f2dc002b020 con 0x7f2dd00f52d0 2024-02-20T12:51:34.913 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-20T12:51:34.913 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":1,"fsid":"680ee126-cfee-11ee-95bb-87774f69a715","modified":"2024-02-20T12:49:44.699059Z","created":"2024-02-20T12:49:44.699059Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi064","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:3300","nonce":0},{"type":"v1","addr":"172.21.15.64:6789","nonce":0}]},"addr":"172.21.15.64:6789/0","public_addr":"172.21.15.64:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-20T12:51:34.915 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:34.914+0000 7f2dd5358700 1 -- 172.21.15.71:0/2680665168 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f2db80310e0 msgr2=0x7f2db80335a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:51:34.915 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:34.914+0000 7f2dd5358700 1 --2- 172.21.15.71:0/2680665168 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f2db80310e0 0x7f2db80335a0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 rx=0x7f2dc40096a0 tx=0x7f2dc4006b40).stop 2024-02-20T12:51:34.915 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:34.914+0000 7f2dd5358700 1 -- 172.21.15.71:0/2680665168 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2dd00f52d0 msgr2=0x7f2dd00fd2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:51:34.915 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:34.914+0000 7f2dd5358700 1 --2- 172.21.15.71:0/2680665168 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2dd00f52d0 0x7f2dd00fd2b0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 rx=0x7f2dc00089f0 tx=0x7f2dc000b040).stop 2024-02-20T12:51:34.915 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:34.914+0000 7f2dd5358700 1 -- 172.21.15.71:0/2680665168 shutdown_connections 2024-02-20T12:51:34.915 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:34.914+0000 7f2dd5358700 1 --2- 172.21.15.71:0/2680665168 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f2db80310e0 0x7f2db80335a0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:51:34.915 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:34.914+0000 7f2dd5358700 1 --2- 172.21.15.71:0/2680665168 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2dd00f52d0 0x7f2dd00fd2b0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:51:34.915 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:34.914+0000 7f2dd5358700 1 -- 172.21.15.71:0/2680665168 >> 172.21.15.71:0/2680665168 conn(0x7f2dd00f0a30 msgr2=0x7f2dd00f1550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:51:34.915 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:34.914+0000 7f2dd5358700 1 -- 172.21.15.71:0/2680665168 shutdown_connections 2024-02-20T12:51:34.916 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:34.914+0000 7f2dd5358700 1 -- 172.21.15.71:0/2680665168 wait complete. 2024-02-20T12:51:34.916 INFO:teuthology.orchestra.run.smithi071.stderr:dumped monmap epoch 1 2024-02-20T12:51:35.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:51:34 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:51:33.706430+0000 mgr.smithi064.vriyor (mgr.14156) 22 : cluster [DBG] pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:51:36.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:51:35 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:51:34.913368+0000 mon.smithi064 (mon.0) 108 : audit [DBG] from='client.? 172.21.15.71:0/2680665168' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-20T12:51:36.569 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-20T12:51:36.569 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/daemon-base:latest-pacific shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 680ee126-cfee-11ee-95bb-87774f69a715 -- ceph mon dump -f json 2024-02-20T12:51:37.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:51:36 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:51:35.706773+0000 mgr.smithi064.vriyor (mgr.14156) 23 : cluster [DBG] pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:51:38.369 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:38.368+0000 7f44563c0700 1 -- 172.21.15.71:0/3045241269 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f44500f6590 msgr2=0x7f44500f69b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:51:38.369 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:38.368+0000 7f44563c0700 1 --2- 172.21.15.71:0/3045241269 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f44500f6590 0x7f44500f69b0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 rx=0x7f4438004760 tx=0x7f4438009ad0).stop 2024-02-20T12:51:38.369 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:38.368+0000 7f44563c0700 1 -- 172.21.15.71:0/3045241269 shutdown_connections 2024-02-20T12:51:38.369 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:38.368+0000 7f44563c0700 1 --2- 172.21.15.71:0/3045241269 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f44500f6590 0x7f44500f69b0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:51:38.370 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:38.368+0000 7f44563c0700 1 -- 172.21.15.71:0/3045241269 >> 172.21.15.71:0/3045241269 conn(0x7f44500f1cd0 msgr2=0x7f44500f4110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:51:38.370 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:38.368+0000 7f44563c0700 1 -- 172.21.15.71:0/3045241269 shutdown_connections 2024-02-20T12:51:38.370 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:38.368+0000 7f44563c0700 1 -- 172.21.15.71:0/3045241269 wait complete. 2024-02-20T12:51:38.370 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:38.369+0000 7f44563c0700 1 Processor -- start 2024-02-20T12:51:38.370 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:38.369+0000 7f44563c0700 1 -- start start 2024-02-20T12:51:38.372 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:38.369+0000 7f44563c0700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f44500f6590 0x7f44501067c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:51:38.372 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:38.369+0000 7f44563c0700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4450106cb0 con 0x7f44500f6590 2024-02-20T12:51:38.372 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:38.369+0000 7f444ffff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f44500f6590 0x7f44501067c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:51:38.372 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:38.369+0000 7f444ffff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f44500f6590 0x7f44501067c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.71:37886/0 (socket says 172.21.15.71:37886) 2024-02-20T12:51:38.372 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:38.369+0000 7f444ffff700 1 -- 172.21.15.71:0/4155637700 learned_addr learned my addr 172.21.15.71:0/4155637700 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-02-20T12:51:38.372 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:38.370+0000 7f444ffff700 1 -- 172.21.15.71:0/4155637700 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4438009d70 con 0x7f44500f6590 2024-02-20T12:51:38.372 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:38.370+0000 7f444ffff700 1 --2- 172.21.15.71:0/4155637700 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f44500f6590 0x7f44501067c0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 rx=0x7f4438004170 tx=0x7f4438009ad0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:51:38.372 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:38.370+0000 7f444cff9700 1 -- 172.21.15.71:0/4155637700 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f443800e040 con 0x7f44500f6590 2024-02-20T12:51:38.372 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:38.370+0000 7f44563c0700 1 -- 172.21.15.71:0/4155637700 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4450106eb0 con 0x7f44500f6590 2024-02-20T12:51:38.372 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:38.370+0000 7f444cff9700 1 -- 172.21.15.71:0/4155637700 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(25 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f4438008ef0 con 0x7f44500f6590 2024-02-20T12:51:38.372 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:38.370+0000 7f444cff9700 1 -- 172.21.15.71:0/4155637700 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4438014c90 con 0x7f44500f6590 2024-02-20T12:51:38.373 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:38.370+0000 7f44563c0700 1 -- 172.21.15.71:0/4155637700 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4450107340 con 0x7f44500f6590 2024-02-20T12:51:38.373 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:38.371+0000 7f444cff9700 1 -- 172.21.15.71:0/4155637700 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 37151+0+0 (secure 0 0 0) 0x7f44380283f0 con 0x7f44500f6590 2024-02-20T12:51:38.373 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:38.371+0000 7f444cff9700 1 --2- 172.21.15.71:0/4155637700 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f443c031090 0x7f443c033550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:51:38.373 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:38.371+0000 7f444cff9700 1 -- 172.21.15.71:0/4155637700 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1065+0+0 (secure 0 0 0) 0x7f4438042300 con 0x7f44500f6590 2024-02-20T12:51:38.373 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:38.371+0000 7f44563c0700 1 -- 172.21.15.71:0/4155637700 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4450045bb0 con 0x7f44500f6590 2024-02-20T12:51:38.373 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:38.371+0000 7f444f7fe700 1 --2- 172.21.15.71:0/4155637700 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f443c031090 0x7f443c033550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:51:38.373 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:38.371+0000 7f444f7fe700 1 --2- 172.21.15.71:0/4155637700 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f443c031090 0x7f443c033550 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 rx=0x7f44400096a0 tx=0x7f4440006b40).ready entity=mgr.14156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:51:38.376 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:38.376+0000 7f444cff9700 1 -- 172.21.15.71:0/4155637700 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+153237 (secure 0 0 0) 0x7f4438012b20 con 0x7f44500f6590 2024-02-20T12:51:38.584 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:38.584+0000 7f44563c0700 1 -- 172.21.15.71:0/4155637700 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f44500fc170 con 0x7f44500f6590 2024-02-20T12:51:38.585 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:38.585+0000 7f444cff9700 1 -- 172.21.15.71:0/4155637700 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7f4438012b20 con 0x7f44500f6590 2024-02-20T12:51:38.585 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-20T12:51:38.585 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":1,"fsid":"680ee126-cfee-11ee-95bb-87774f69a715","modified":"2024-02-20T12:49:44.699059Z","created":"2024-02-20T12:49:44.699059Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi064","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:3300","nonce":0},{"type":"v1","addr":"172.21.15.64:6789","nonce":0}]},"addr":"172.21.15.64:6789/0","public_addr":"172.21.15.64:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-20T12:51:38.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:38.586+0000 7f44563c0700 1 -- 172.21.15.71:0/4155637700 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f443c031090 msgr2=0x7f443c033550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:51:38.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:38.586+0000 7f44563c0700 1 --2- 172.21.15.71:0/4155637700 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f443c031090 0x7f443c033550 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 rx=0x7f44400096a0 tx=0x7f4440006b40).stop 2024-02-20T12:51:38.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:38.586+0000 7f44563c0700 1 -- 172.21.15.71:0/4155637700 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f44500f6590 msgr2=0x7f44501067c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:51:38.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:38.586+0000 7f44563c0700 1 --2- 172.21.15.71:0/4155637700 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f44500f6590 0x7f44501067c0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 rx=0x7f4438004170 tx=0x7f4438009ad0).stop 2024-02-20T12:51:38.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:38.586+0000 7f44563c0700 1 -- 172.21.15.71:0/4155637700 shutdown_connections 2024-02-20T12:51:38.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:38.586+0000 7f44563c0700 1 --2- 172.21.15.71:0/4155637700 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f443c031090 0x7f443c033550 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:51:38.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:38.586+0000 7f44563c0700 1 --2- 172.21.15.71:0/4155637700 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f44500f6590 0x7f44501067c0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:51:38.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:38.586+0000 7f44563c0700 1 -- 172.21.15.71:0/4155637700 >> 172.21.15.71:0/4155637700 conn(0x7f44500f1cd0 msgr2=0x7f44500f27c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:51:38.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:38.586+0000 7f44563c0700 1 -- 172.21.15.71:0/4155637700 shutdown_connections 2024-02-20T12:51:38.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:38.586+0000 7f44563c0700 1 -- 172.21.15.71:0/4155637700 wait complete. 2024-02-20T12:51:38.588 INFO:teuthology.orchestra.run.smithi071.stderr:dumped monmap epoch 1 2024-02-20T12:51:39.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:51:38 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:51:37.707100+0000 mgr.smithi064.vriyor (mgr.14156) 24 : cluster [DBG] pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:51:39.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:51:38 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:51:38.585306+0000 mon.smithi064 (mon.0) 109 : audit [DBG] from='client.? 172.21.15.71:0/4155637700' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-20T12:51:40.159 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-20T12:51:40.160 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/daemon-base:latest-pacific shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 680ee126-cfee-11ee-95bb-87774f69a715 -- ceph mon dump -f json 2024-02-20T12:51:41.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:51:40 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:51:39.707537+0000 mgr.smithi064.vriyor (mgr.14156) 25 : cluster [DBG] pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:51:41.991 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:41.988+0000 7f1d1bd09700 1 -- 172.21.15.71:0/3642618172 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1d140f5480 msgr2=0x7f1d140f58a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:51:41.991 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:41.988+0000 7f1d1bd09700 1 --2- 172.21.15.71:0/3642618172 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1d140f5480 0x7f1d140f58a0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 rx=0x7f1d04009fb0 tx=0x7f1d04009b30).stop 2024-02-20T12:51:41.991 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:41.988+0000 7f1d1bd09700 1 -- 172.21.15.71:0/3642618172 shutdown_connections 2024-02-20T12:51:41.991 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:41.988+0000 7f1d1bd09700 1 --2- 172.21.15.71:0/3642618172 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1d140f5480 0x7f1d140f58a0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:51:41.991 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:41.988+0000 7f1d1bd09700 1 -- 172.21.15.71:0/3642618172 >> 172.21.15.71:0/3642618172 conn(0x7f1d140f0be0 msgr2=0x7f1d140f3000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:51:41.991 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:41.989+0000 7f1d1bd09700 1 -- 172.21.15.71:0/3642618172 shutdown_connections 2024-02-20T12:51:41.991 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:41.989+0000 7f1d1bd09700 1 -- 172.21.15.71:0/3642618172 wait complete. 2024-02-20T12:51:41.991 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:41.989+0000 7f1d1bd09700 1 Processor -- start 2024-02-20T12:51:41.991 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:41.989+0000 7f1d1bd09700 1 -- start start 2024-02-20T12:51:41.991 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:41.989+0000 7f1d1bd09700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1d140f5480 0x7f1d140fd460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:51:41.992 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:41.989+0000 7f1d1bd09700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d140fb8e0 con 0x7f1d140f5480 2024-02-20T12:51:41.992 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:41.989+0000 7f1d19aa5700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1d140f5480 0x7f1d140fd460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:51:41.992 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:41.989+0000 7f1d19aa5700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1d140f5480 0x7f1d140fd460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.71:37896/0 (socket says 172.21.15.71:37896) 2024-02-20T12:51:41.992 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:41.989+0000 7f1d19aa5700 1 -- 172.21.15.71:0/854393143 learned_addr learned my addr 172.21.15.71:0/854393143 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-02-20T12:51:41.992 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:41.990+0000 7f1d19aa5700 1 -- 172.21.15.71:0/854393143 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d04005040 con 0x7f1d140f5480 2024-02-20T12:51:41.992 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:41.990+0000 7f1d19aa5700 1 --2- 172.21.15.71:0/854393143 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1d140f5480 0x7f1d140fd460 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 rx=0x7f1d040089f0 tx=0x7f1d0400b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:51:41.992 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:41.990+0000 7f1d0a7fc700 1 -- 172.21.15.71:0/854393143 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1d04009320 con 0x7f1d140f5480 2024-02-20T12:51:41.992 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:41.990+0000 7f1d0a7fc700 1 -- 172.21.15.71:0/854393143 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(25 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f1d04013ec0 con 0x7f1d140f5480 2024-02-20T12:51:41.992 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:41.990+0000 7f1d1bd09700 1 -- 172.21.15.71:0/854393143 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d140fba80 con 0x7f1d140f5480 2024-02-20T12:51:41.992 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:41.990+0000 7f1d0a7fc700 1 -- 172.21.15.71:0/854393143 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1d04014d30 con 0x7f1d140f5480 2024-02-20T12:51:41.992 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:41.990+0000 7f1d1bd09700 1 -- 172.21.15.71:0/854393143 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d140fbf10 con 0x7f1d140f5480 2024-02-20T12:51:41.993 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:41.991+0000 7f1d0a7fc700 1 -- 172.21.15.71:0/854393143 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 37151+0+0 (secure 0 0 0) 0x7f1d04014420 con 0x7f1d140f5480 2024-02-20T12:51:41.993 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:41.991+0000 7f1d1bd09700 1 -- 172.21.15.71:0/854393143 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d140591e0 con 0x7f1d140f5480 2024-02-20T12:51:41.993 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:41.991+0000 7f1d0a7fc700 1 --2- 172.21.15.71:0/854393143 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f1d00031090 0x7f1d00033550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:51:41.993 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:41.991+0000 7f1d0a7fc700 1 -- 172.21.15.71:0/854393143 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1065+0+0 (secure 0 0 0) 0x7f1d040424c0 con 0x7f1d140f5480 2024-02-20T12:51:41.993 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:41.991+0000 7f1d192a4700 1 --2- 172.21.15.71:0/854393143 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f1d00031090 0x7f1d00033550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:51:41.994 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:41.992+0000 7f1d192a4700 1 --2- 172.21.15.71:0/854393143 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f1d00031090 0x7f1d00033550 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0x7f1d100096a0 tx=0x7f1d10006b40).ready entity=mgr.14156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:51:41.996 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:41.996+0000 7f1d0a7fc700 1 -- 172.21.15.71:0/854393143 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+153237 (secure 0 0 0) 0x7f1d0401d980 con 0x7f1d140f5480 2024-02-20T12:51:42.203 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:42.202+0000 7f1d1bd09700 1 -- 172.21.15.71:0/854393143 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f1d140faf70 con 0x7f1d140f5480 2024-02-20T12:51:42.203 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:42.203+0000 7f1d0a7fc700 1 -- 172.21.15.71:0/854393143 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7f1d0401d380 con 0x7f1d140f5480 2024-02-20T12:51:42.203 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-20T12:51:42.204 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":1,"fsid":"680ee126-cfee-11ee-95bb-87774f69a715","modified":"2024-02-20T12:49:44.699059Z","created":"2024-02-20T12:49:44.699059Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi064","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:3300","nonce":0},{"type":"v1","addr":"172.21.15.64:6789","nonce":0}]},"addr":"172.21.15.64:6789/0","public_addr":"172.21.15.64:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-20T12:51:42.205 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:42.204+0000 7f1d1bd09700 1 -- 172.21.15.71:0/854393143 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f1d00031090 msgr2=0x7f1d00033550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:51:42.205 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:42.204+0000 7f1d1bd09700 1 --2- 172.21.15.71:0/854393143 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f1d00031090 0x7f1d00033550 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0x7f1d100096a0 tx=0x7f1d10006b40).stop 2024-02-20T12:51:42.205 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:42.204+0000 7f1d1bd09700 1 -- 172.21.15.71:0/854393143 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1d140f5480 msgr2=0x7f1d140fd460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:51:42.205 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:42.204+0000 7f1d1bd09700 1 --2- 172.21.15.71:0/854393143 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1d140f5480 0x7f1d140fd460 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 rx=0x7f1d040089f0 tx=0x7f1d0400b040).stop 2024-02-20T12:51:42.205 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:42.204+0000 7f1d1bd09700 1 -- 172.21.15.71:0/854393143 shutdown_connections 2024-02-20T12:51:42.205 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:42.204+0000 7f1d1bd09700 1 --2- 172.21.15.71:0/854393143 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f1d00031090 0x7f1d00033550 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:51:42.205 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:42.204+0000 7f1d1bd09700 1 --2- 172.21.15.71:0/854393143 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1d140f5480 0x7f1d140fd460 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:51:42.205 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:42.204+0000 7f1d1bd09700 1 -- 172.21.15.71:0/854393143 >> 172.21.15.71:0/854393143 conn(0x7f1d140f0be0 msgr2=0x7f1d1405ff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:51:42.205 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:42.204+0000 7f1d1bd09700 1 -- 172.21.15.71:0/854393143 shutdown_connections 2024-02-20T12:51:42.206 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:42.204+0000 7f1d1bd09700 1 -- 172.21.15.71:0/854393143 wait complete. 2024-02-20T12:51:42.206 INFO:teuthology.orchestra.run.smithi071.stderr:dumped monmap epoch 1 2024-02-20T12:51:43.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:51:42 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:51:41.707825+0000 mgr.smithi064.vriyor (mgr.14156) 26 : cluster [DBG] pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:51:43.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:51:42 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:51:42.203452+0000 mon.smithi064 (mon.0) 110 : audit [DBG] from='client.? 172.21.15.71:0/854393143' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-20T12:51:43.856 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-20T12:51:43.857 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/daemon-base:latest-pacific shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 680ee126-cfee-11ee-95bb-87774f69a715 -- ceph mon dump -f json 2024-02-20T12:51:45.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:51:44 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:51:43.708281+0000 mgr.smithi064.vriyor (mgr.14156) 27 : cluster [DBG] pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:51:45.799 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:45.797+0000 7faa72565700 1 -- 172.21.15.71:0/2116866049 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faa6c0f6590 msgr2=0x7faa6c0f69b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:51:45.799 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:45.797+0000 7faa72565700 1 --2- 172.21.15.71:0/2116866049 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faa6c0f6590 0x7faa6c0f69b0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 rx=0x7faa5c004760 tx=0x7faa5c009ad0).stop 2024-02-20T12:51:45.799 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:45.797+0000 7faa72565700 1 -- 172.21.15.71:0/2116866049 shutdown_connections 2024-02-20T12:51:45.799 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:45.797+0000 7faa72565700 1 --2- 172.21.15.71:0/2116866049 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faa6c0f6590 0x7faa6c0f69b0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:51:45.799 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:45.797+0000 7faa72565700 1 -- 172.21.15.71:0/2116866049 >> 172.21.15.71:0/2116866049 conn(0x7faa6c0f1cd0 msgr2=0x7faa6c0f4110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:51:45.799 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:45.797+0000 7faa72565700 1 -- 172.21.15.71:0/2116866049 shutdown_connections 2024-02-20T12:51:45.799 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:45.797+0000 7faa72565700 1 -- 172.21.15.71:0/2116866049 wait complete. 2024-02-20T12:51:45.800 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:45.798+0000 7faa72565700 1 Processor -- start 2024-02-20T12:51:45.800 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:45.798+0000 7faa72565700 1 -- start start 2024-02-20T12:51:45.800 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:45.798+0000 7faa72565700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faa6c0f6590 0x7faa6c108a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:51:45.800 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:45.798+0000 7faa72565700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa6c108f00 con 0x7faa6c0f6590 2024-02-20T12:51:45.800 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:45.798+0000 7faa6bfff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faa6c0f6590 0x7faa6c108a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:51:45.800 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:45.798+0000 7faa6bfff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faa6c0f6590 0x7faa6c108a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.71:37904/0 (socket says 172.21.15.71:37904) 2024-02-20T12:51:45.800 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:45.798+0000 7faa6bfff700 1 -- 172.21.15.71:0/3145505332 learned_addr learned my addr 172.21.15.71:0/3145505332 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-02-20T12:51:45.800 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:45.799+0000 7faa6bfff700 1 -- 172.21.15.71:0/3145505332 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa5c009d70 con 0x7faa6c0f6590 2024-02-20T12:51:45.802 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:45.799+0000 7faa6bfff700 1 --2- 172.21.15.71:0/3145505332 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faa6c0f6590 0x7faa6c108a10 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 rx=0x7faa5c004170 tx=0x7faa5c009ad0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:51:45.802 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:45.799+0000 7faa68ff9700 1 -- 172.21.15.71:0/3145505332 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7faa5c00e040 con 0x7faa6c0f6590 2024-02-20T12:51:45.802 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:45.799+0000 7faa68ff9700 1 -- 172.21.15.71:0/3145505332 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(25 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7faa5c008ef0 con 0x7faa6c0f6590 2024-02-20T12:51:45.803 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:45.799+0000 7faa72565700 1 -- 172.21.15.71:0/3145505332 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa6c109100 con 0x7faa6c0f6590 2024-02-20T12:51:45.803 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:45.799+0000 7faa68ff9700 1 -- 172.21.15.71:0/3145505332 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7faa5c014c90 con 0x7faa6c0f6590 2024-02-20T12:51:45.803 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:45.799+0000 7faa72565700 1 -- 172.21.15.71:0/3145505332 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa6c109590 con 0x7faa6c0f6590 2024-02-20T12:51:45.803 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:45.800+0000 7faa68ff9700 1 -- 172.21.15.71:0/3145505332 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 37151+0+0 (secure 0 0 0) 0x7faa5c014420 con 0x7faa6c0f6590 2024-02-20T12:51:45.803 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:45.800+0000 7faa68ff9700 1 --2- 172.21.15.71:0/3145505332 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7faa540310e0 0x7faa540335a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:51:45.803 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:45.800+0000 7faa72565700 1 -- 172.21.15.71:0/3145505332 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa6c045bb0 con 0x7faa6c0f6590 2024-02-20T12:51:45.803 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:45.800+0000 7faa68ff9700 1 -- 172.21.15.71:0/3145505332 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1065+0+0 (secure 0 0 0) 0x7faa5c0422b0 con 0x7faa6c0f6590 2024-02-20T12:51:45.803 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:45.800+0000 7faa6b7fe700 1 --2- 172.21.15.71:0/3145505332 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7faa540310e0 0x7faa540335a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:51:45.803 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:45.801+0000 7faa6b7fe700 1 --2- 172.21.15.71:0/3145505332 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7faa540310e0 0x7faa540335a0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7faa600096a0 tx=0x7faa60006b40).ready entity=mgr.14156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:51:45.806 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:45.805+0000 7faa68ff9700 1 -- 172.21.15.71:0/3145505332 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+153237 (secure 0 0 0) 0x7faa5c026300 con 0x7faa6c0f6590 2024-02-20T12:51:46.012 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:46.011+0000 7faa72565700 1 -- 172.21.15.71:0/3145505332 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7faa6c10a340 con 0x7faa6c0f6590 2024-02-20T12:51:46.013 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:46.012+0000 7faa68ff9700 1 -- 172.21.15.71:0/3145505332 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7faa5c012b20 con 0x7faa6c0f6590 2024-02-20T12:51:46.013 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-20T12:51:46.013 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":1,"fsid":"680ee126-cfee-11ee-95bb-87774f69a715","modified":"2024-02-20T12:49:44.699059Z","created":"2024-02-20T12:49:44.699059Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi064","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:3300","nonce":0},{"type":"v1","addr":"172.21.15.64:6789","nonce":0}]},"addr":"172.21.15.64:6789/0","public_addr":"172.21.15.64:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-20T12:51:46.014 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:46.014+0000 7faa72565700 1 -- 172.21.15.71:0/3145505332 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7faa540310e0 msgr2=0x7faa540335a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:51:46.015 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:46.014+0000 7faa72565700 1 --2- 172.21.15.71:0/3145505332 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7faa540310e0 0x7faa540335a0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7faa600096a0 tx=0x7faa60006b40).stop 2024-02-20T12:51:46.015 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:46.014+0000 7faa72565700 1 -- 172.21.15.71:0/3145505332 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faa6c0f6590 msgr2=0x7faa6c108a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:51:46.015 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:46.014+0000 7faa72565700 1 --2- 172.21.15.71:0/3145505332 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faa6c0f6590 0x7faa6c108a10 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 rx=0x7faa5c004170 tx=0x7faa5c009ad0).stop 2024-02-20T12:51:46.015 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:46.014+0000 7faa72565700 1 -- 172.21.15.71:0/3145505332 shutdown_connections 2024-02-20T12:51:46.015 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:46.014+0000 7faa72565700 1 --2- 172.21.15.71:0/3145505332 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7faa540310e0 0x7faa540335a0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:51:46.015 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:46.014+0000 7faa72565700 1 --2- 172.21.15.71:0/3145505332 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faa6c0f6590 0x7faa6c108a10 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:51:46.015 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:46.014+0000 7faa72565700 1 -- 172.21.15.71:0/3145505332 >> 172.21.15.71:0/3145505332 conn(0x7faa6c0f1cd0 msgr2=0x7faa6c0f2780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:51:46.015 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:46.014+0000 7faa72565700 1 -- 172.21.15.71:0/3145505332 shutdown_connections 2024-02-20T12:51:46.015 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:46.014+0000 7faa72565700 1 -- 172.21.15.71:0/3145505332 wait complete. 2024-02-20T12:51:46.016 INFO:teuthology.orchestra.run.smithi071.stderr:dumped monmap epoch 1 2024-02-20T12:51:47.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:51:46 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:51:45.708764+0000 mgr.smithi064.vriyor (mgr.14156) 28 : cluster [DBG] pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:51:47.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:51:46 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:51:46.013099+0000 mon.smithi064 (mon.0) 111 : audit [DBG] from='client.? 172.21.15.71:0/3145505332' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-20T12:51:47.625 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-20T12:51:47.625 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/daemon-base:latest-pacific shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 680ee126-cfee-11ee-95bb-87774f69a715 -- ceph mon dump -f json 2024-02-20T12:51:49.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:51:48 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:51:47.709152+0000 mgr.smithi064.vriyor (mgr.14156) 29 : cluster [DBG] pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:51:49.461 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:49.457+0000 7f52b1e19700 1 -- 172.21.15.71:0/1972404229 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f52ac060300 msgr2=0x7f52ac0f91c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:51:49.462 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:49.457+0000 7f52b1e19700 1 --2- 172.21.15.71:0/1972404229 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f52ac060300 0x7f52ac0f91c0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 rx=0x7f52a0009fb0 tx=0x7f52a0009b30).stop 2024-02-20T12:51:49.462 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:49.457+0000 7f52b1e19700 1 -- 172.21.15.71:0/1972404229 shutdown_connections 2024-02-20T12:51:49.462 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:49.457+0000 7f52b1e19700 1 --2- 172.21.15.71:0/1972404229 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f52ac060300 0x7f52ac0f91c0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:51:49.462 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:49.457+0000 7f52b1e19700 1 -- 172.21.15.71:0/1972404229 >> 172.21.15.71:0/1972404229 conn(0x7f52ac0f0be0 msgr2=0x7f52ac0f3000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:51:49.462 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:49.457+0000 7f52b1e19700 1 -- 172.21.15.71:0/1972404229 shutdown_connections 2024-02-20T12:51:49.462 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:49.457+0000 7f52b1e19700 1 -- 172.21.15.71:0/1972404229 wait complete. 2024-02-20T12:51:49.462 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:49.457+0000 7f52b1e19700 1 Processor -- start 2024-02-20T12:51:49.462 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:49.458+0000 7f52b1e19700 1 -- start start 2024-02-20T12:51:49.463 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:49.458+0000 7f52b1e19700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f52ac060300 0x7f52ac108980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:51:49.463 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:49.458+0000 7f52b1e19700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52ac108e70 con 0x7f52ac060300 2024-02-20T12:51:49.463 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:49.458+0000 7f52ab7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f52ac060300 0x7f52ac108980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:51:49.463 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:49.458+0000 7f52ab7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f52ac060300 0x7f52ac108980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.71:42126/0 (socket says 172.21.15.71:42126) 2024-02-20T12:51:49.463 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:49.458+0000 7f52ab7fe700 1 -- 172.21.15.71:0/1424112038 learned_addr learned my addr 172.21.15.71:0/1424112038 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-02-20T12:51:49.463 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:49.458+0000 7f52ab7fe700 1 -- 172.21.15.71:0/1424112038 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52a0005040 con 0x7f52ac060300 2024-02-20T12:51:49.463 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:49.458+0000 7f52ab7fe700 1 --2- 172.21.15.71:0/1424112038 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f52ac060300 0x7f52ac108980 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 rx=0x7f52a0009f80 tx=0x7f52a000b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:51:49.464 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:49.459+0000 7f5293fff700 1 -- 172.21.15.71:0/1424112038 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f52a0009320 con 0x7f52ac060300 2024-02-20T12:51:49.464 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:49.459+0000 7f5293fff700 1 -- 172.21.15.71:0/1424112038 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(25 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f52a0013680 con 0x7f52ac060300 2024-02-20T12:51:49.464 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:49.459+0000 7f52b1e19700 1 -- 172.21.15.71:0/1424112038 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52ac109070 con 0x7f52ac060300 2024-02-20T12:51:49.464 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:49.459+0000 7f5293fff700 1 -- 172.21.15.71:0/1424112038 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f52a001eda0 con 0x7f52ac060300 2024-02-20T12:51:49.464 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:49.459+0000 7f52b1e19700 1 -- 172.21.15.71:0/1424112038 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52ac109500 con 0x7f52ac060300 2024-02-20T12:51:49.464 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:49.459+0000 7f5293fff700 1 -- 172.21.15.71:0/1424112038 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 37151+0+0 (secure 0 0 0) 0x7f52a0014420 con 0x7f52ac060300 2024-02-20T12:51:49.464 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:49.460+0000 7f5293fff700 1 --2- 172.21.15.71:0/1424112038 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f5294031090 0x7f5294033550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:51:49.464 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:49.460+0000 7f5293fff700 1 -- 172.21.15.71:0/1424112038 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1065+0+0 (secure 0 0 0) 0x7f52a0042330 con 0x7f52ac060300 2024-02-20T12:51:49.464 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:49.460+0000 7f52b1e19700 1 -- 172.21.15.71:0/1424112038 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52ac0591e0 con 0x7f52ac060300 2024-02-20T12:51:49.464 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:49.460+0000 7f52aaffd700 1 --2- 172.21.15.71:0/1424112038 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f5294031090 0x7f5294033550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:51:49.465 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:49.460+0000 7f52aaffd700 1 --2- 172.21.15.71:0/1424112038 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f5294031090 0x7f5294033550 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7f529c0096a0 tx=0x7f529c006b40).ready entity=mgr.14156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:51:49.465 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:49.465+0000 7f5293fff700 1 -- 172.21.15.71:0/1424112038 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+153237 (secure 0 0 0) 0x7f52a001c910 con 0x7f52ac060300 2024-02-20T12:51:49.673 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:49.673+0000 7f52b1e19700 1 -- 172.21.15.71:0/1424112038 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f52ac045bb0 con 0x7f52ac060300 2024-02-20T12:51:49.674 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:49.673+0000 7f5293fff700 1 -- 172.21.15.71:0/1424112038 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7f52a001c320 con 0x7f52ac060300 2024-02-20T12:51:49.674 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-20T12:51:49.674 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":1,"fsid":"680ee126-cfee-11ee-95bb-87774f69a715","modified":"2024-02-20T12:49:44.699059Z","created":"2024-02-20T12:49:44.699059Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi064","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:3300","nonce":0},{"type":"v1","addr":"172.21.15.64:6789","nonce":0}]},"addr":"172.21.15.64:6789/0","public_addr":"172.21.15.64:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-20T12:51:49.676 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:49.675+0000 7f52b1e19700 1 -- 172.21.15.71:0/1424112038 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f5294031090 msgr2=0x7f5294033550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:51:49.676 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:49.675+0000 7f52b1e19700 1 --2- 172.21.15.71:0/1424112038 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f5294031090 0x7f5294033550 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7f529c0096a0 tx=0x7f529c006b40).stop 2024-02-20T12:51:49.676 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:49.675+0000 7f52b1e19700 1 -- 172.21.15.71:0/1424112038 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f52ac060300 msgr2=0x7f52ac108980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:51:49.676 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:49.675+0000 7f52b1e19700 1 --2- 172.21.15.71:0/1424112038 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f52ac060300 0x7f52ac108980 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 rx=0x7f52a0009f80 tx=0x7f52a000b040).stop 2024-02-20T12:51:49.676 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:49.675+0000 7f52b1e19700 1 -- 172.21.15.71:0/1424112038 shutdown_connections 2024-02-20T12:51:49.676 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:49.675+0000 7f52b1e19700 1 --2- 172.21.15.71:0/1424112038 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f5294031090 0x7f5294033550 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:51:49.676 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:49.675+0000 7f52b1e19700 1 --2- 172.21.15.71:0/1424112038 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f52ac060300 0x7f52ac108980 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:51:49.676 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:49.675+0000 7f52b1e19700 1 -- 172.21.15.71:0/1424112038 >> 172.21.15.71:0/1424112038 conn(0x7f52ac0f0be0 msgr2=0x7f52ac0f16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:51:49.676 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:49.675+0000 7f52b1e19700 1 -- 172.21.15.71:0/1424112038 shutdown_connections 2024-02-20T12:51:49.676 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:49.675+0000 7f52b1e19700 1 -- 172.21.15.71:0/1424112038 wait complete. 2024-02-20T12:51:49.677 INFO:teuthology.orchestra.run.smithi071.stderr:dumped monmap epoch 1 2024-02-20T12:51:50.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:51:49 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:51:49.674021+0000 mon.smithi064 (mon.0) 112 : audit [DBG] from='client.? 172.21.15.71:0/1424112038' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-20T12:51:51.265 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-20T12:51:51.266 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/daemon-base:latest-pacific shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 680ee126-cfee-11ee-95bb-87774f69a715 -- ceph mon dump -f json 2024-02-20T12:51:51.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:51:50 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:51:49.709462+0000 mgr.smithi064.vriyor (mgr.14156) 30 : cluster [DBG] pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:51:53.074 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:53.073+0000 7effee845700 1 -- 172.21.15.71:0/1380195016 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7effe80f6590 msgr2=0x7effe80f69b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:51:53.074 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:53.073+0000 7effee845700 1 --2- 172.21.15.71:0/1380195016 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7effe80f6590 0x7effe80f69b0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 rx=0x7effd8004760 tx=0x7effd8009ad0).stop 2024-02-20T12:51:53.074 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:53.073+0000 7effee845700 1 -- 172.21.15.71:0/1380195016 shutdown_connections 2024-02-20T12:51:53.075 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:53.073+0000 7effee845700 1 --2- 172.21.15.71:0/1380195016 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7effe80f6590 0x7effe80f69b0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:51:53.075 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:53.073+0000 7effee845700 1 -- 172.21.15.71:0/1380195016 >> 172.21.15.71:0/1380195016 conn(0x7effe80f1cd0 msgr2=0x7effe80f4110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:51:53.075 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:53.073+0000 7effee845700 1 -- 172.21.15.71:0/1380195016 shutdown_connections 2024-02-20T12:51:53.075 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:53.073+0000 7effee845700 1 -- 172.21.15.71:0/1380195016 wait complete. 2024-02-20T12:51:53.075 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:53.074+0000 7effee845700 1 Processor -- start 2024-02-20T12:51:53.077 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:53.074+0000 7effee845700 1 -- start start 2024-02-20T12:51:53.077 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:53.074+0000 7effee845700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7effe80f6590 0x7effe81067c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:51:53.077 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:53.074+0000 7effee845700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effe8106cb0 con 0x7effe80f6590 2024-02-20T12:51:53.077 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:53.075+0000 7effe7fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7effe80f6590 0x7effe81067c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:51:53.077 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:53.075+0000 7effe7fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7effe80f6590 0x7effe81067c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.71:42148/0 (socket says 172.21.15.71:42148) 2024-02-20T12:51:53.077 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:53.075+0000 7effe7fff700 1 -- 172.21.15.71:0/3725405905 learned_addr learned my addr 172.21.15.71:0/3725405905 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-02-20T12:51:53.077 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:53.075+0000 7effe7fff700 1 -- 172.21.15.71:0/3725405905 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effd8009d70 con 0x7effe80f6590 2024-02-20T12:51:53.077 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:53.075+0000 7effe7fff700 1 --2- 172.21.15.71:0/3725405905 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7effe80f6590 0x7effe81067c0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 rx=0x7effd8004170 tx=0x7effd8009ad0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:51:53.078 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:53.075+0000 7effe4ff9700 1 -- 172.21.15.71:0/3725405905 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7effd800e040 con 0x7effe80f6590 2024-02-20T12:51:53.078 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:53.075+0000 7effe4ff9700 1 -- 172.21.15.71:0/3725405905 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(25 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7effd8008ef0 con 0x7effe80f6590 2024-02-20T12:51:53.078 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:53.075+0000 7effe4ff9700 1 -- 172.21.15.71:0/3725405905 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7effd8014c90 con 0x7effe80f6590 2024-02-20T12:51:53.078 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:53.075+0000 7effee845700 1 -- 172.21.15.71:0/3725405905 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7effe8106eb0 con 0x7effe80f6590 2024-02-20T12:51:53.078 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:53.076+0000 7effee845700 1 -- 172.21.15.71:0/3725405905 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7effe8107340 con 0x7effe80f6590 2024-02-20T12:51:53.078 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:53.076+0000 7effe4ff9700 1 -- 172.21.15.71:0/3725405905 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 37151+0+0 (secure 0 0 0) 0x7effd8014420 con 0x7effe80f6590 2024-02-20T12:51:53.078 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:53.076+0000 7effe4ff9700 1 --2- 172.21.15.71:0/3725405905 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7effd0031130 0x7effd00335f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:51:53.078 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:53.076+0000 7effee845700 1 -- 172.21.15.71:0/3725405905 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7effe8045bb0 con 0x7effe80f6590 2024-02-20T12:51:53.078 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:53.076+0000 7effe4ff9700 1 -- 172.21.15.71:0/3725405905 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1065+0+0 (secure 0 0 0) 0x7effd80422b0 con 0x7effe80f6590 2024-02-20T12:51:53.078 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:53.077+0000 7effe77fe700 1 --2- 172.21.15.71:0/3725405905 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7effd0031130 0x7effd00335f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:51:53.079 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:53.077+0000 7effe77fe700 1 --2- 172.21.15.71:0/3725405905 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7effd0031130 0x7effd00335f0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7effdc0096a0 tx=0x7effdc006b40).ready entity=mgr.14156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:51:53.081 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:53.081+0000 7effe4ff9700 1 -- 172.21.15.71:0/3725405905 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+153237 (secure 0 0 0) 0x7effd8026300 con 0x7effe80f6590 2024-02-20T12:51:53.287 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:53.285+0000 7effee845700 1 -- 172.21.15.71:0/3725405905 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7effe8108140 con 0x7effe80f6590 2024-02-20T12:51:53.287 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:53.286+0000 7effe4ff9700 1 -- 172.21.15.71:0/3725405905 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7effd8012b20 con 0x7effe80f6590 2024-02-20T12:51:53.288 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-20T12:51:53.288 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":1,"fsid":"680ee126-cfee-11ee-95bb-87774f69a715","modified":"2024-02-20T12:49:44.699059Z","created":"2024-02-20T12:49:44.699059Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi064","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:3300","nonce":0},{"type":"v1","addr":"172.21.15.64:6789","nonce":0}]},"addr":"172.21.15.64:6789/0","public_addr":"172.21.15.64:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-20T12:51:53.289 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:53.288+0000 7effee845700 1 -- 172.21.15.71:0/3725405905 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7effd0031130 msgr2=0x7effd00335f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:51:53.289 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:53.288+0000 7effee845700 1 --2- 172.21.15.71:0/3725405905 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7effd0031130 0x7effd00335f0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7effdc0096a0 tx=0x7effdc006b40).stop 2024-02-20T12:51:53.290 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:53.288+0000 7effee845700 1 -- 172.21.15.71:0/3725405905 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7effe80f6590 msgr2=0x7effe81067c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:51:53.290 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:53.288+0000 7effee845700 1 --2- 172.21.15.71:0/3725405905 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7effe80f6590 0x7effe81067c0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 rx=0x7effd8004170 tx=0x7effd8009ad0).stop 2024-02-20T12:51:53.290 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:53.288+0000 7effee845700 1 -- 172.21.15.71:0/3725405905 shutdown_connections 2024-02-20T12:51:53.290 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:53.288+0000 7effee845700 1 --2- 172.21.15.71:0/3725405905 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7effd0031130 0x7effd00335f0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:51:53.290 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:53.288+0000 7effee845700 1 --2- 172.21.15.71:0/3725405905 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7effe80f6590 0x7effe81067c0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:51:53.290 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:53.288+0000 7effee845700 1 -- 172.21.15.71:0/3725405905 >> 172.21.15.71:0/3725405905 conn(0x7effe80f1cd0 msgr2=0x7effe80f27c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:51:53.290 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:53.288+0000 7effee845700 1 -- 172.21.15.71:0/3725405905 shutdown_connections 2024-02-20T12:51:53.290 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:53.288+0000 7effee845700 1 -- 172.21.15.71:0/3725405905 wait complete. 2024-02-20T12:51:53.290 INFO:teuthology.orchestra.run.smithi071.stderr:dumped monmap epoch 1 2024-02-20T12:51:53.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:51:52 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:51:51.709895+0000 mgr.smithi064.vriyor (mgr.14156) 31 : cluster [DBG] pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:51:54.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:51:53 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:51:53.287947+0000 mon.smithi064 (mon.0) 113 : audit [DBG] from='client.? 172.21.15.71:0/3725405905' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-20T12:51:54.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:51:53 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:51:53.843122+0000 mon.smithi064 (mon.0) 114 : audit [INF] from='mgr.14156 172.21.15.64:0/3284974785' entity='mgr.smithi064.vriyor' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi064.vriyor/mirror_snapshot_schedule"}]: dispatch 2024-02-20T12:51:54.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:51:53 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:51:53.844350+0000 mon.smithi064 (mon.0) 115 : audit [INF] from='mgr.14156 172.21.15.64:0/3284974785' entity='mgr.smithi064.vriyor' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi064.vriyor/trash_purge_schedule"}]: dispatch 2024-02-20T12:51:55.010 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-20T12:51:55.010 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/daemon-base:latest-pacific shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 680ee126-cfee-11ee-95bb-87774f69a715 -- ceph mon dump -f json 2024-02-20T12:51:55.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:51:54 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:51:53.710350+0000 mgr.smithi064.vriyor (mgr.14156) 32 : cluster [DBG] pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:51:56.877 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:56.874+0000 7fe38cf67700 1 -- 172.21.15.71:0/2510921013 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe3880f5480 msgr2=0x7fe3880f58a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:51:56.877 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:56.874+0000 7fe38cf67700 1 --2- 172.21.15.71:0/2510921013 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe3880f5480 0x7fe3880f58a0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 rx=0x7fe370009fb0 tx=0x7fe370009b30).stop 2024-02-20T12:51:56.877 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:56.874+0000 7fe38cf67700 1 -- 172.21.15.71:0/2510921013 shutdown_connections 2024-02-20T12:51:56.877 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:56.874+0000 7fe38cf67700 1 --2- 172.21.15.71:0/2510921013 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe3880f5480 0x7fe3880f58a0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:51:56.877 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:56.874+0000 7fe38cf67700 1 -- 172.21.15.71:0/2510921013 >> 172.21.15.71:0/2510921013 conn(0x7fe3880f0be0 msgr2=0x7fe3880f3000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:51:56.877 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:56.874+0000 7fe38cf67700 1 -- 172.21.15.71:0/2510921013 shutdown_connections 2024-02-20T12:51:56.877 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:56.874+0000 7fe38cf67700 1 -- 172.21.15.71:0/2510921013 wait complete. 2024-02-20T12:51:56.878 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:56.874+0000 7fe38cf67700 1 Processor -- start 2024-02-20T12:51:56.878 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:56.874+0000 7fe38cf67700 1 -- start start 2024-02-20T12:51:56.878 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:56.874+0000 7fe38cf67700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe3880f5480 0x7fe3881089e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:51:56.878 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:56.874+0000 7fe38cf67700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe388108ed0 con 0x7fe3880f5480 2024-02-20T12:51:56.878 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:56.875+0000 7fe38659c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe3880f5480 0x7fe3881089e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:51:56.878 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:56.875+0000 7fe38659c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe3880f5480 0x7fe3881089e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.71:42166/0 (socket says 172.21.15.71:42166) 2024-02-20T12:51:56.878 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:56.875+0000 7fe38659c700 1 -- 172.21.15.71:0/3650860473 learned_addr learned my addr 172.21.15.71:0/3650860473 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-02-20T12:51:56.878 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:56.875+0000 7fe38659c700 1 -- 172.21.15.71:0/3650860473 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe370005040 con 0x7fe3880f5480 2024-02-20T12:51:56.878 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:56.875+0000 7fe38659c700 1 --2- 172.21.15.71:0/3650860473 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe3880f5480 0x7fe3881089e0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 rx=0x7fe370000c00 tx=0x7fe37000b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:51:56.878 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:56.875+0000 7fe37effd700 1 -- 172.21.15.71:0/3650860473 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe3700095f0 con 0x7fe3880f5480 2024-02-20T12:51:56.878 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:56.875+0000 7fe38cf67700 1 -- 172.21.15.71:0/3650860473 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3881090d0 con 0x7fe3880f5480 2024-02-20T12:51:56.878 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:56.875+0000 7fe37effd700 1 -- 172.21.15.71:0/3650860473 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(25 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7fe370007e50 con 0x7fe3880f5480 2024-02-20T12:51:56.878 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:56.875+0000 7fe37effd700 1 -- 172.21.15.71:0/3650860473 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe370014e70 con 0x7fe3880f5480 2024-02-20T12:51:56.878 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:56.876+0000 7fe38cf67700 1 -- 172.21.15.71:0/3650860473 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe388109560 con 0x7fe3880f5480 2024-02-20T12:51:56.879 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:56.876+0000 7fe37effd700 1 -- 172.21.15.71:0/3650860473 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 37151+0+0 (secure 0 0 0) 0x7fe370025070 con 0x7fe3880f5480 2024-02-20T12:51:56.879 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:56.877+0000 7fe38cf67700 1 -- 172.21.15.71:0/3650860473 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe388045bb0 con 0x7fe3880f5480 2024-02-20T12:51:56.880 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:56.877+0000 7fe37effd700 1 --2- 172.21.15.71:0/3650860473 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fe374031090 0x7fe374033550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:51:56.880 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:56.877+0000 7fe37effd700 1 -- 172.21.15.71:0/3650860473 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1065+0+0 (secure 0 0 0) 0x7fe370041d20 con 0x7fe3880f5480 2024-02-20T12:51:56.880 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:56.877+0000 7fe385d9b700 1 --2- 172.21.15.71:0/3650860473 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fe374031090 0x7fe374033550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:51:56.880 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:56.877+0000 7fe385d9b700 1 --2- 172.21.15.71:0/3650860473 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fe374031090 0x7fe374033550 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7fe3780096a0 tx=0x7fe378006b40).ready entity=mgr.14156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:51:56.883 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:56.881+0000 7fe37effd700 1 -- 172.21.15.71:0/3650860473 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+153237 (secure 0 0 0) 0x7fe37000f8d0 con 0x7fe3880f5480 2024-02-20T12:51:57.091 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:57.089+0000 7fe38cf67700 1 -- 172.21.15.71:0/3650860473 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fe38810a370 con 0x7fe3880f5480 2024-02-20T12:51:57.091 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:57.090+0000 7fe37effd700 1 -- 172.21.15.71:0/3650860473 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7fe37000f300 con 0x7fe3880f5480 2024-02-20T12:51:57.092 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-20T12:51:57.092 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":1,"fsid":"680ee126-cfee-11ee-95bb-87774f69a715","modified":"2024-02-20T12:49:44.699059Z","created":"2024-02-20T12:49:44.699059Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi064","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:3300","nonce":0},{"type":"v1","addr":"172.21.15.64:6789","nonce":0}]},"addr":"172.21.15.64:6789/0","public_addr":"172.21.15.64:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-20T12:51:57.093 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:57.092+0000 7fe38cf67700 1 -- 172.21.15.71:0/3650860473 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fe374031090 msgr2=0x7fe374033550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:51:57.094 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:57.092+0000 7fe38cf67700 1 --2- 172.21.15.71:0/3650860473 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fe374031090 0x7fe374033550 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7fe3780096a0 tx=0x7fe378006b40).stop 2024-02-20T12:51:57.094 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:57.092+0000 7fe38cf67700 1 -- 172.21.15.71:0/3650860473 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe3880f5480 msgr2=0x7fe3881089e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:51:57.094 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:57.092+0000 7fe38cf67700 1 --2- 172.21.15.71:0/3650860473 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe3880f5480 0x7fe3881089e0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 rx=0x7fe370000c00 tx=0x7fe37000b040).stop 2024-02-20T12:51:57.094 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:57.092+0000 7fe38cf67700 1 -- 172.21.15.71:0/3650860473 shutdown_connections 2024-02-20T12:51:57.094 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:57.092+0000 7fe38cf67700 1 --2- 172.21.15.71:0/3650860473 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fe374031090 0x7fe374033550 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:51:57.094 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:57.092+0000 7fe38cf67700 1 --2- 172.21.15.71:0/3650860473 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe3880f5480 0x7fe3881089e0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:51:57.094 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:57.092+0000 7fe38cf67700 1 -- 172.21.15.71:0/3650860473 >> 172.21.15.71:0/3650860473 conn(0x7fe3880f0be0 msgr2=0x7fe38805ff40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:51:57.094 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:57.092+0000 7fe38cf67700 1 -- 172.21.15.71:0/3650860473 shutdown_connections 2024-02-20T12:51:57.094 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:51:57.092+0000 7fe38cf67700 1 -- 172.21.15.71:0/3650860473 wait complete. 2024-02-20T12:51:57.094 INFO:teuthology.orchestra.run.smithi071.stderr:dumped monmap epoch 1 2024-02-20T12:51:57.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:51:56 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:51:55.710810+0000 mgr.smithi064.vriyor (mgr.14156) 33 : cluster [DBG] pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:51:58.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:51:57 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:51:57.091918+0000 mon.smithi064 (mon.0) 116 : audit [DBG] from='client.? 172.21.15.71:0/3650860473' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-20T12:51:58.667 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-20T12:51:58.668 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/daemon-base:latest-pacific shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 680ee126-cfee-11ee-95bb-87774f69a715 -- ceph mon dump -f json 2024-02-20T12:51:59.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:51:58 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:51:57.711155+0000 mgr.smithi064.vriyor (mgr.14156) 34 : cluster [DBG] pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:52:00.636 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:00.592+0000 7f5092cdf700 1 -- 172.21.15.71:0/1457556114 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f508c0f63e0 msgr2=0x7f508c0f6800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:52:00.637 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:00.593+0000 7f5092cdf700 1 --2- 172.21.15.71:0/1457556114 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f508c0f63e0 0x7f508c0f6800 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 rx=0x7f507c009fb0 tx=0x7f507c009b30).stop 2024-02-20T12:52:00.637 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:00.593+0000 7f5092cdf700 1 -- 172.21.15.71:0/1457556114 shutdown_connections 2024-02-20T12:52:00.637 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:00.593+0000 7f5092cdf700 1 --2- 172.21.15.71:0/1457556114 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f508c0f63e0 0x7f508c0f6800 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:52:00.637 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:00.593+0000 7f5092cdf700 1 -- 172.21.15.71:0/1457556114 >> 172.21.15.71:0/1457556114 conn(0x7f508c0f1b40 msgr2=0x7f508c0f3f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:52:00.637 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:00.593+0000 7f5092cdf700 1 -- 172.21.15.71:0/1457556114 shutdown_connections 2024-02-20T12:52:00.637 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:00.593+0000 7f5092cdf700 1 -- 172.21.15.71:0/1457556114 wait complete. 2024-02-20T12:52:00.637 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:00.593+0000 7f5092cdf700 1 Processor -- start 2024-02-20T12:52:00.637 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:00.593+0000 7f5092cdf700 1 -- start start 2024-02-20T12:52:00.637 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:00.594+0000 7f5092cdf700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f508c0f63e0 0x7f508c106620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:52:00.637 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:00.594+0000 7f5092cdf700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f508c106b10 con 0x7f508c0f63e0 2024-02-20T12:52:00.637 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:00.594+0000 7f5091cdd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f508c0f63e0 0x7f508c106620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:52:00.638 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:00.594+0000 7f5091cdd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f508c0f63e0 0x7f508c106620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.71:41406/0 (socket says 172.21.15.71:41406) 2024-02-20T12:52:00.638 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:00.594+0000 7f5091cdd700 1 -- 172.21.15.71:0/3393576838 learned_addr learned my addr 172.21.15.71:0/3393576838 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-02-20T12:52:00.638 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:00.594+0000 7f5091cdd700 1 -- 172.21.15.71:0/3393576838 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f507c005040 con 0x7f508c0f63e0 2024-02-20T12:52:00.638 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:00.594+0000 7f5091cdd700 1 --2- 172.21.15.71:0/3393576838 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f508c0f63e0 0x7f508c106620 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 rx=0x7f507c000c00 tx=0x7f507c00b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:52:00.638 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:00.594+0000 7f50827fc700 1 -- 172.21.15.71:0/3393576838 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f507c0095f0 con 0x7f508c0f63e0 2024-02-20T12:52:00.638 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:00.594+0000 7f50827fc700 1 -- 172.21.15.71:0/3393576838 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(25 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f507c007e50 con 0x7f508c0f63e0 2024-02-20T12:52:00.638 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:00.594+0000 7f50827fc700 1 -- 172.21.15.71:0/3393576838 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f507c014e70 con 0x7f508c0f63e0 2024-02-20T12:52:00.638 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:00.594+0000 7f5092cdf700 1 -- 172.21.15.71:0/3393576838 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f508c106d10 con 0x7f508c0f63e0 2024-02-20T12:52:00.638 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:00.594+0000 7f5092cdf700 1 -- 172.21.15.71:0/3393576838 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f508c1071a0 con 0x7f508c0f63e0 2024-02-20T12:52:00.638 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:00.595+0000 7f50827fc700 1 -- 172.21.15.71:0/3393576838 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 37151+0+0 (secure 0 0 0) 0x7f507c025070 con 0x7f508c0f63e0 2024-02-20T12:52:00.638 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:00.595+0000 7f50827fc700 1 --2- 172.21.15.71:0/3393576838 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f5078031090 0x7f5078033550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:52:00.638 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:00.595+0000 7f50827fc700 1 -- 172.21.15.71:0/3393576838 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1065+0+0 (secure 0 0 0) 0x7f507c041d40 con 0x7f508c0f63e0 2024-02-20T12:52:00.638 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:00.595+0000 7f5092cdf700 1 -- 172.21.15.71:0/3393576838 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f508c045bb0 con 0x7f508c0f63e0 2024-02-20T12:52:00.638 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:00.596+0000 7f50914dc700 1 --2- 172.21.15.71:0/3393576838 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f5078031090 0x7f5078033550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:52:00.638 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:00.596+0000 7f50914dc700 1 --2- 172.21.15.71:0/3393576838 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f5078031090 0x7f5078033550 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0x7f50880096a0 tx=0x7f5088006b40).ready entity=mgr.14156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:52:00.639 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:00.600+0000 7f50827fc700 1 -- 172.21.15.71:0/3393576838 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+153237 (secure 0 0 0) 0x7f507c0079c0 con 0x7f508c0f63e0 2024-02-20T12:52:00.811 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:00.809+0000 7f5092cdf700 1 -- 172.21.15.71:0/3393576838 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f508c107fa0 con 0x7f508c0f63e0 2024-02-20T12:52:00.811 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:00.810+0000 7f50827fc700 1 -- 172.21.15.71:0/3393576838 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7f507c00f300 con 0x7f508c0f63e0 2024-02-20T12:52:00.812 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-20T12:52:00.812 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":1,"fsid":"680ee126-cfee-11ee-95bb-87774f69a715","modified":"2024-02-20T12:49:44.699059Z","created":"2024-02-20T12:49:44.699059Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi064","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:3300","nonce":0},{"type":"v1","addr":"172.21.15.64:6789","nonce":0}]},"addr":"172.21.15.64:6789/0","public_addr":"172.21.15.64:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-20T12:52:00.814 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:00.812+0000 7f5092cdf700 1 -- 172.21.15.71:0/3393576838 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f5078031090 msgr2=0x7f5078033550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:52:00.814 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:00.812+0000 7f5092cdf700 1 --2- 172.21.15.71:0/3393576838 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f5078031090 0x7f5078033550 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0x7f50880096a0 tx=0x7f5088006b40).stop 2024-02-20T12:52:00.814 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:00.812+0000 7f5092cdf700 1 -- 172.21.15.71:0/3393576838 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f508c0f63e0 msgr2=0x7f508c106620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:52:00.814 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:00.812+0000 7f5092cdf700 1 --2- 172.21.15.71:0/3393576838 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f508c0f63e0 0x7f508c106620 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 rx=0x7f507c000c00 tx=0x7f507c00b040).stop 2024-02-20T12:52:00.814 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:00.812+0000 7f5092cdf700 1 -- 172.21.15.71:0/3393576838 shutdown_connections 2024-02-20T12:52:00.814 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:00.812+0000 7f5092cdf700 1 --2- 172.21.15.71:0/3393576838 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f5078031090 0x7f5078033550 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:52:00.814 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:00.812+0000 7f5092cdf700 1 --2- 172.21.15.71:0/3393576838 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f508c0f63e0 0x7f508c106620 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:52:00.814 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:00.812+0000 7f5092cdf700 1 -- 172.21.15.71:0/3393576838 >> 172.21.15.71:0/3393576838 conn(0x7f508c0f1b40 msgr2=0x7f508c0f2660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:52:00.814 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:00.812+0000 7f5092cdf700 1 -- 172.21.15.71:0/3393576838 shutdown_connections 2024-02-20T12:52:00.814 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:00.812+0000 7f5092cdf700 1 -- 172.21.15.71:0/3393576838 wait complete. 2024-02-20T12:52:00.814 INFO:teuthology.orchestra.run.smithi071.stderr:dumped monmap epoch 1 2024-02-20T12:52:01.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:52:00 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:51:59.711516+0000 mgr.smithi064.vriyor (mgr.14156) 35 : cluster [DBG] pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:52:01.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:52:00 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:52:00.811692+0000 mon.smithi064 (mon.0) 117 : audit [DBG] from='client.? 172.21.15.71:0/3393576838' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-20T12:52:02.522 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-20T12:52:02.522 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/daemon-base:latest-pacific shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 680ee126-cfee-11ee-95bb-87774f69a715 -- ceph mon dump -f json 2024-02-20T12:52:03.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:52:02 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:52:01.711915+0000 mgr.smithi064.vriyor (mgr.14156) 36 : cluster [DBG] pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:52:04.398 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:04.394+0000 7f340165e700 1 -- 172.21.15.71:0/3919872801 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f33fc0f63e0 msgr2=0x7f33fc0f6800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:52:04.398 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:04.394+0000 7f340165e700 1 --2- 172.21.15.71:0/3919872801 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f33fc0f63e0 0x7f33fc0f6800 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 rx=0x7f33ec009fb0 tx=0x7f33ec009b30).stop 2024-02-20T12:52:04.398 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:04.395+0000 7f340165e700 1 -- 172.21.15.71:0/3919872801 shutdown_connections 2024-02-20T12:52:04.398 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:04.395+0000 7f340165e700 1 --2- 172.21.15.71:0/3919872801 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f33fc0f63e0 0x7f33fc0f6800 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:52:04.399 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:04.395+0000 7f340165e700 1 -- 172.21.15.71:0/3919872801 >> 172.21.15.71:0/3919872801 conn(0x7f33fc0f1b40 msgr2=0x7f33fc0f3f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:52:04.399 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:04.395+0000 7f340165e700 1 -- 172.21.15.71:0/3919872801 shutdown_connections 2024-02-20T12:52:04.399 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:04.395+0000 7f340165e700 1 -- 172.21.15.71:0/3919872801 wait complete. 2024-02-20T12:52:04.399 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:04.395+0000 7f340165e700 1 Processor -- start 2024-02-20T12:52:04.399 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:04.395+0000 7f340165e700 1 -- start start 2024-02-20T12:52:04.399 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:04.395+0000 7f340165e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f33fc0f63e0 0x7f33fc106620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:52:04.399 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:04.395+0000 7f340165e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33fc106b10 con 0x7f33fc0f63e0 2024-02-20T12:52:04.399 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:04.396+0000 7f33fbfff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f33fc0f63e0 0x7f33fc106620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:52:04.399 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:04.396+0000 7f33fbfff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f33fc0f63e0 0x7f33fc106620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.71:41412/0 (socket says 172.21.15.71:41412) 2024-02-20T12:52:04.399 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:04.396+0000 7f33fbfff700 1 -- 172.21.15.71:0/1017724958 learned_addr learned my addr 172.21.15.71:0/1017724958 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-02-20T12:52:04.399 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:04.396+0000 7f33fbfff700 1 -- 172.21.15.71:0/1017724958 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33ec005040 con 0x7f33fc0f63e0 2024-02-20T12:52:04.399 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:04.396+0000 7f33fbfff700 1 --2- 172.21.15.71:0/1017724958 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f33fc0f63e0 0x7f33fc106620 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 rx=0x7f33ec0089f0 tx=0x7f33ec00b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:52:04.399 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:04.396+0000 7f33f8ff9700 1 -- 172.21.15.71:0/1017724958 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f33ec009320 con 0x7f33fc0f63e0 2024-02-20T12:52:04.399 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:04.396+0000 7f33f8ff9700 1 -- 172.21.15.71:0/1017724958 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(25 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f33ec013ec0 con 0x7f33fc0f63e0 2024-02-20T12:52:04.400 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:04.396+0000 7f340165e700 1 -- 172.21.15.71:0/1017724958 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33fc106d10 con 0x7f33fc0f63e0 2024-02-20T12:52:04.400 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:04.396+0000 7f33f8ff9700 1 -- 172.21.15.71:0/1017724958 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f33ec014d30 con 0x7f33fc0f63e0 2024-02-20T12:52:04.400 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:04.396+0000 7f340165e700 1 -- 172.21.15.71:0/1017724958 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33fc1071a0 con 0x7f33fc0f63e0 2024-02-20T12:52:04.401 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:04.397+0000 7f340165e700 1 -- 172.21.15.71:0/1017724958 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33fc045bb0 con 0x7f33fc0f63e0 2024-02-20T12:52:04.401 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:04.397+0000 7f33f8ff9700 1 -- 172.21.15.71:0/1017724958 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 37151+0+0 (secure 0 0 0) 0x7f33ec0095f0 con 0x7f33fc0f63e0 2024-02-20T12:52:04.401 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:04.398+0000 7f33f8ff9700 1 --2- 172.21.15.71:0/1017724958 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f33e4031090 0x7f33e4033550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:52:04.401 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:04.398+0000 7f33f8ff9700 1 -- 172.21.15.71:0/1017724958 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1065+0+0 (secure 0 0 0) 0x7f33ec042420 con 0x7f33fc0f63e0 2024-02-20T12:52:04.401 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:04.398+0000 7f33fb7fe700 1 --2- 172.21.15.71:0/1017724958 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f33e4031090 0x7f33e4033550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:52:04.401 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:04.398+0000 7f33fb7fe700 1 --2- 172.21.15.71:0/1017724958 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f33e4031090 0x7f33e4033550 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0x7f33f00096a0 tx=0x7f33f0006b40).ready entity=mgr.14156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:52:04.404 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:04.402+0000 7f33f8ff9700 1 -- 172.21.15.71:0/1017724958 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+153237 (secure 0 0 0) 0x7f33ec01d7f0 con 0x7f33fc0f63e0 2024-02-20T12:52:04.609 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:04.608+0000 7f340165e700 1 -- 172.21.15.71:0/1017724958 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f33fc107fd0 con 0x7f33fc0f63e0 2024-02-20T12:52:04.610 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:04.609+0000 7f33f8ff9700 1 -- 172.21.15.71:0/1017724958 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7f33ec00fe40 con 0x7f33fc0f63e0 2024-02-20T12:52:04.610 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-20T12:52:04.611 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":1,"fsid":"680ee126-cfee-11ee-95bb-87774f69a715","modified":"2024-02-20T12:49:44.699059Z","created":"2024-02-20T12:49:44.699059Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi064","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:3300","nonce":0},{"type":"v1","addr":"172.21.15.64:6789","nonce":0}]},"addr":"172.21.15.64:6789/0","public_addr":"172.21.15.64:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-20T12:52:04.612 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:04.610+0000 7f340165e700 1 -- 172.21.15.71:0/1017724958 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f33e4031090 msgr2=0x7f33e4033550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:52:04.612 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:04.610+0000 7f340165e700 1 --2- 172.21.15.71:0/1017724958 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f33e4031090 0x7f33e4033550 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0x7f33f00096a0 tx=0x7f33f0006b40).stop 2024-02-20T12:52:04.612 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:04.610+0000 7f340165e700 1 -- 172.21.15.71:0/1017724958 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f33fc0f63e0 msgr2=0x7f33fc106620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:52:04.612 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:04.610+0000 7f340165e700 1 --2- 172.21.15.71:0/1017724958 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f33fc0f63e0 0x7f33fc106620 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 rx=0x7f33ec0089f0 tx=0x7f33ec00b040).stop 2024-02-20T12:52:04.612 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:04.610+0000 7f340165e700 1 -- 172.21.15.71:0/1017724958 shutdown_connections 2024-02-20T12:52:04.612 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:04.610+0000 7f340165e700 1 --2- 172.21.15.71:0/1017724958 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f33e4031090 0x7f33e4033550 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:52:04.613 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:04.610+0000 7f340165e700 1 --2- 172.21.15.71:0/1017724958 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f33fc0f63e0 0x7f33fc106620 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:52:04.613 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:04.610+0000 7f340165e700 1 -- 172.21.15.71:0/1017724958 >> 172.21.15.71:0/1017724958 conn(0x7f33fc0f1b40 msgr2=0x7f33fc0f2660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:52:04.613 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:04.610+0000 7f340165e700 1 -- 172.21.15.71:0/1017724958 shutdown_connections 2024-02-20T12:52:04.613 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:04.610+0000 7f340165e700 1 -- 172.21.15.71:0/1017724958 wait complete. 2024-02-20T12:52:04.613 INFO:teuthology.orchestra.run.smithi071.stderr:dumped monmap epoch 1 2024-02-20T12:52:05.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:52:04 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:52:03.712364+0000 mgr.smithi064.vriyor (mgr.14156) 37 : cluster [DBG] pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:52:05.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:52:04 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:52:04.610517+0000 mon.smithi064 (mon.0) 118 : audit [DBG] from='client.? 172.21.15.71:0/1017724958' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-20T12:52:06.144 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-20T12:52:06.145 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/daemon-base:latest-pacific shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 680ee126-cfee-11ee-95bb-87774f69a715 -- ceph mon dump -f json 2024-02-20T12:52:07.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:52:06 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:52:05.712805+0000 mgr.smithi064.vriyor (mgr.14156) 38 : cluster [DBG] pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:52:07.870 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:07.840+0000 7fa1c8216700 1 -- 172.21.15.71:0/2564401563 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa1c00f5480 msgr2=0x7fa1c00f58a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:52:07.870 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:07.840+0000 7fa1c8216700 1 --2- 172.21.15.71:0/2564401563 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa1c00f5480 0x7fa1c00f58a0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 rx=0x7fa1b0009fb0 tx=0x7fa1b0009b30).stop 2024-02-20T12:52:07.870 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:07.840+0000 7fa1c8216700 1 -- 172.21.15.71:0/2564401563 shutdown_connections 2024-02-20T12:52:07.870 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:07.840+0000 7fa1c8216700 1 --2- 172.21.15.71:0/2564401563 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa1c00f5480 0x7fa1c00f58a0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:52:07.870 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:07.840+0000 7fa1c8216700 1 -- 172.21.15.71:0/2564401563 >> 172.21.15.71:0/2564401563 conn(0x7fa1c00f0be0 msgr2=0x7fa1c00f3000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:52:07.871 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:07.840+0000 7fa1c8216700 1 -- 172.21.15.71:0/2564401563 shutdown_connections 2024-02-20T12:52:07.871 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:07.840+0000 7fa1c8216700 1 -- 172.21.15.71:0/2564401563 wait complete. 2024-02-20T12:52:07.871 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:07.841+0000 7fa1c8216700 1 Processor -- start 2024-02-20T12:52:07.871 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:07.841+0000 7fa1c8216700 1 -- start start 2024-02-20T12:52:07.871 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:07.841+0000 7fa1c8216700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa1c00f5480 0x7fa1c01067e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:52:07.871 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:07.841+0000 7fa1c8216700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1c0106cd0 con 0x7fa1c00f5480 2024-02-20T12:52:07.871 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:07.841+0000 7fa1c5fb2700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa1c00f5480 0x7fa1c01067e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:52:07.871 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:07.841+0000 7fa1c5fb2700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa1c00f5480 0x7fa1c01067e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.71:48694/0 (socket says 172.21.15.71:48694) 2024-02-20T12:52:07.871 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:07.841+0000 7fa1c5fb2700 1 -- 172.21.15.71:0/3729859455 learned_addr learned my addr 172.21.15.71:0/3729859455 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-02-20T12:52:07.871 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:07.842+0000 7fa1c5fb2700 1 -- 172.21.15.71:0/3729859455 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1b0005040 con 0x7fa1c00f5480 2024-02-20T12:52:07.871 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:07.842+0000 7fa1c5fb2700 1 --2- 172.21.15.71:0/3729859455 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa1c00f5480 0x7fa1c01067e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 rx=0x7fa1b00089f0 tx=0x7fa1b000b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:52:07.871 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:07.842+0000 7fa1b67fc700 1 -- 172.21.15.71:0/3729859455 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa1b0009320 con 0x7fa1c00f5480 2024-02-20T12:52:07.872 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:07.842+0000 7fa1c8216700 1 -- 172.21.15.71:0/3729859455 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1c0106ed0 con 0x7fa1c00f5480 2024-02-20T12:52:07.872 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:07.842+0000 7fa1b67fc700 1 -- 172.21.15.71:0/3729859455 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(25 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7fa1b0013ec0 con 0x7fa1c00f5480 2024-02-20T12:52:07.872 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:07.842+0000 7fa1b67fc700 1 -- 172.21.15.71:0/3729859455 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa1b0014d30 con 0x7fa1c00f5480 2024-02-20T12:52:07.872 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:07.842+0000 7fa1c8216700 1 -- 172.21.15.71:0/3729859455 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1c0107360 con 0x7fa1c00f5480 2024-02-20T12:52:07.872 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:07.843+0000 7fa1b67fc700 1 -- 172.21.15.71:0/3729859455 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 37151+0+0 (secure 0 0 0) 0x7fa1b0014420 con 0x7fa1c00f5480 2024-02-20T12:52:07.872 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:07.843+0000 7fa1b67fc700 1 --2- 172.21.15.71:0/3729859455 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fa1ac031090 0x7fa1ac033550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:52:07.872 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:07.843+0000 7fa1c8216700 1 -- 172.21.15.71:0/3729859455 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1c0045bb0 con 0x7fa1c00f5480 2024-02-20T12:52:07.872 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:07.843+0000 7fa1b67fc700 1 -- 172.21.15.71:0/3729859455 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1065+0+0 (secure 0 0 0) 0x7fa1b00423e0 con 0x7fa1c00f5480 2024-02-20T12:52:07.872 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:07.843+0000 7fa1c57b1700 1 --2- 172.21.15.71:0/3729859455 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fa1ac031090 0x7fa1ac033550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:52:07.872 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:07.843+0000 7fa1c57b1700 1 --2- 172.21.15.71:0/3729859455 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fa1ac031090 0x7fa1ac033550 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0x7fa1bc0096a0 tx=0x7fa1bc006b40).ready entity=mgr.14156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:52:07.872 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:07.848+0000 7fa1b67fc700 1 -- 172.21.15.71:0/3729859455 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+153237 (secure 0 0 0) 0x7fa1b001d7f0 con 0x7fa1c00f5480 2024-02-20T12:52:08.054 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:08.053+0000 7fa1c8216700 1 -- 172.21.15.71:0/3729859455 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fa1c0108190 con 0x7fa1c00f5480 2024-02-20T12:52:08.056 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:08.054+0000 7fa1b67fc700 1 -- 172.21.15.71:0/3729859455 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7fa1b001d7f0 con 0x7fa1c00f5480 2024-02-20T12:52:08.056 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-20T12:52:08.056 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":1,"fsid":"680ee126-cfee-11ee-95bb-87774f69a715","modified":"2024-02-20T12:49:44.699059Z","created":"2024-02-20T12:49:44.699059Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi064","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:3300","nonce":0},{"type":"v1","addr":"172.21.15.64:6789","nonce":0}]},"addr":"172.21.15.64:6789/0","public_addr":"172.21.15.64:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-20T12:52:08.057 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:08.055+0000 7fa1c8216700 1 -- 172.21.15.71:0/3729859455 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fa1ac031090 msgr2=0x7fa1ac033550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:52:08.057 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:08.055+0000 7fa1c8216700 1 --2- 172.21.15.71:0/3729859455 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fa1ac031090 0x7fa1ac033550 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0x7fa1bc0096a0 tx=0x7fa1bc006b40).stop 2024-02-20T12:52:08.058 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:08.055+0000 7fa1c8216700 1 -- 172.21.15.71:0/3729859455 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa1c00f5480 msgr2=0x7fa1c01067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:52:08.058 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:08.055+0000 7fa1c8216700 1 --2- 172.21.15.71:0/3729859455 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa1c00f5480 0x7fa1c01067e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 rx=0x7fa1b00089f0 tx=0x7fa1b000b040).stop 2024-02-20T12:52:08.058 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:08.055+0000 7fa1c8216700 1 -- 172.21.15.71:0/3729859455 shutdown_connections 2024-02-20T12:52:08.058 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:08.055+0000 7fa1c8216700 1 --2- 172.21.15.71:0/3729859455 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fa1ac031090 0x7fa1ac033550 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:52:08.058 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:08.055+0000 7fa1c8216700 1 --2- 172.21.15.71:0/3729859455 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa1c00f5480 0x7fa1c01067e0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:52:08.058 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:08.055+0000 7fa1c8216700 1 -- 172.21.15.71:0/3729859455 >> 172.21.15.71:0/3729859455 conn(0x7fa1c00f0be0 msgr2=0x7fa1c005ff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:52:08.058 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:08.055+0000 7fa1c8216700 1 -- 172.21.15.71:0/3729859455 shutdown_connections 2024-02-20T12:52:08.059 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:08.055+0000 7fa1c8216700 1 -- 172.21.15.71:0/3729859455 wait complete. 2024-02-20T12:52:08.059 INFO:teuthology.orchestra.run.smithi071.stderr:dumped monmap epoch 1 2024-02-20T12:52:09.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:52:08 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:52:07.713210+0000 mgr.smithi064.vriyor (mgr.14156) 39 : cluster [DBG] pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:52:09.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:52:08 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:52:08.055401+0000 mon.smithi064 (mon.0) 119 : audit [DBG] from='client.? 172.21.15.71:0/3729859455' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-20T12:52:09.795 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-20T12:52:09.795 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/daemon-base:latest-pacific shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 680ee126-cfee-11ee-95bb-87774f69a715 -- ceph mon dump -f json 2024-02-20T12:52:11.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:52:10 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:52:09.713510+0000 mgr.smithi064.vriyor (mgr.14156) 40 : cluster [DBG] pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:52:11.639 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:11.636+0000 7f50b4db9700 1 -- 172.21.15.71:0/1909349214 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f50b00f54b0 msgr2=0x7f50b00f58d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:52:11.639 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:11.636+0000 7f50b4db9700 1 --2- 172.21.15.71:0/1909349214 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f50b00f54b0 0x7f50b00f58d0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 rx=0x7f50a0009fb0 tx=0x7f50a0009b30).stop 2024-02-20T12:52:11.639 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:11.636+0000 7f50b4db9700 1 -- 172.21.15.71:0/1909349214 shutdown_connections 2024-02-20T12:52:11.639 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:11.636+0000 7f50b4db9700 1 --2- 172.21.15.71:0/1909349214 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f50b00f54b0 0x7f50b00f58d0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:52:11.639 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:11.636+0000 7f50b4db9700 1 -- 172.21.15.71:0/1909349214 >> 172.21.15.71:0/1909349214 conn(0x7f50b00f0c10 msgr2=0x7f50b00f3030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:52:11.639 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:11.636+0000 7f50b4db9700 1 -- 172.21.15.71:0/1909349214 shutdown_connections 2024-02-20T12:52:11.639 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:11.636+0000 7f50b4db9700 1 -- 172.21.15.71:0/1909349214 wait complete. 2024-02-20T12:52:11.639 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:11.636+0000 7f50b4db9700 1 Processor -- start 2024-02-20T12:52:11.640 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:11.637+0000 7f50b4db9700 1 -- start start 2024-02-20T12:52:11.640 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:11.637+0000 7f50b4db9700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f50b00f54b0 0x7f50b0109520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:52:11.640 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:11.637+0000 7f50b4db9700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50b0060300 con 0x7f50b00f54b0 2024-02-20T12:52:11.640 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:11.637+0000 7f50ae59c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f50b00f54b0 0x7f50b0109520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:52:11.640 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:11.637+0000 7f50ae59c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f50b00f54b0 0x7f50b0109520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.71:48716/0 (socket says 172.21.15.71:48716) 2024-02-20T12:52:11.640 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:11.637+0000 7f50ae59c700 1 -- 172.21.15.71:0/3111336459 learned_addr learned my addr 172.21.15.71:0/3111336459 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-02-20T12:52:11.640 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:11.637+0000 7f50ae59c700 1 -- 172.21.15.71:0/3111336459 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50a0005040 con 0x7f50b00f54b0 2024-02-20T12:52:11.640 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:11.637+0000 7f50ae59c700 1 --2- 172.21.15.71:0/3111336459 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f50b00f54b0 0x7f50b0109520 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 rx=0x7f50a00089f0 tx=0x7f50a000b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:52:11.640 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:11.637+0000 7f509effd700 1 -- 172.21.15.71:0/3111336459 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f50a0009320 con 0x7f50b00f54b0 2024-02-20T12:52:11.640 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:11.637+0000 7f50b4db9700 1 -- 172.21.15.71:0/3111336459 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50b0060500 con 0x7f50b00f54b0 2024-02-20T12:52:11.640 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:11.637+0000 7f509effd700 1 -- 172.21.15.71:0/3111336459 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(25 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f50a0013ec0 con 0x7f50b00f54b0 2024-02-20T12:52:11.640 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:11.638+0000 7f509effd700 1 -- 172.21.15.71:0/3111336459 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f50a0014d30 con 0x7f50b00f54b0 2024-02-20T12:52:11.640 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:11.638+0000 7f50b4db9700 1 -- 172.21.15.71:0/3111336459 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50b0109bd0 con 0x7f50b00f54b0 2024-02-20T12:52:11.641 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:11.638+0000 7f509effd700 1 -- 172.21.15.71:0/3111336459 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 37151+0+0 (secure 0 0 0) 0x7f50a00263a0 con 0x7f50b00f54b0 2024-02-20T12:52:11.641 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:11.638+0000 7f509effd700 1 --2- 172.21.15.71:0/3111336459 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f50980310e0 0x7f50980335a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:52:11.641 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:11.638+0000 7f50b4db9700 1 -- 172.21.15.71:0/3111336459 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50b00591e0 con 0x7f50b00f54b0 2024-02-20T12:52:11.642 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:11.638+0000 7f509effd700 1 -- 172.21.15.71:0/3111336459 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1065+0+0 (secure 0 0 0) 0x7f50a00423a0 con 0x7f50b00f54b0 2024-02-20T12:52:11.642 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:11.639+0000 7f50add9b700 1 --2- 172.21.15.71:0/3111336459 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f50980310e0 0x7f50980335a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:52:11.642 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:11.639+0000 7f50add9b700 1 --2- 172.21.15.71:0/3111336459 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f50980310e0 0x7f50980335a0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0x7f50a40096a0 tx=0x7f50a4006b40).ready entity=mgr.14156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:52:11.645 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:11.643+0000 7f509effd700 1 -- 172.21.15.71:0/3111336459 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+153237 (secure 0 0 0) 0x7f50a0012330 con 0x7f50b00f54b0 2024-02-20T12:52:11.851 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:11.850+0000 7f50b4db9700 1 -- 172.21.15.71:0/3111336459 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f50b0045bb0 con 0x7f50b00f54b0 2024-02-20T12:52:11.852 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:11.851+0000 7f509effd700 1 -- 172.21.15.71:0/3111336459 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7f50a002b020 con 0x7f50b00f54b0 2024-02-20T12:52:11.853 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-20T12:52:11.853 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":1,"fsid":"680ee126-cfee-11ee-95bb-87774f69a715","modified":"2024-02-20T12:49:44.699059Z","created":"2024-02-20T12:49:44.699059Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi064","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:3300","nonce":0},{"type":"v1","addr":"172.21.15.64:6789","nonce":0}]},"addr":"172.21.15.64:6789/0","public_addr":"172.21.15.64:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-20T12:52:11.854 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:11.852+0000 7f50b4db9700 1 -- 172.21.15.71:0/3111336459 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f50980310e0 msgr2=0x7f50980335a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:52:11.854 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:11.852+0000 7f50b4db9700 1 --2- 172.21.15.71:0/3111336459 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f50980310e0 0x7f50980335a0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0x7f50a40096a0 tx=0x7f50a4006b40).stop 2024-02-20T12:52:11.854 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:11.852+0000 7f50b4db9700 1 -- 172.21.15.71:0/3111336459 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f50b00f54b0 msgr2=0x7f50b0109520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:52:11.854 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:11.852+0000 7f50b4db9700 1 --2- 172.21.15.71:0/3111336459 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f50b00f54b0 0x7f50b0109520 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 rx=0x7f50a00089f0 tx=0x7f50a000b040).stop 2024-02-20T12:52:11.854 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:11.852+0000 7f50b4db9700 1 -- 172.21.15.71:0/3111336459 shutdown_connections 2024-02-20T12:52:11.855 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:11.852+0000 7f50b4db9700 1 --2- 172.21.15.71:0/3111336459 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f50980310e0 0x7f50980335a0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:52:11.855 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:11.852+0000 7f50b4db9700 1 --2- 172.21.15.71:0/3111336459 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f50b00f54b0 0x7f50b0109520 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:52:11.855 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:11.852+0000 7f50b4db9700 1 -- 172.21.15.71:0/3111336459 >> 172.21.15.71:0/3111336459 conn(0x7f50b00f0c10 msgr2=0x7f50b00f1700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:52:11.855 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:11.852+0000 7f50b4db9700 1 -- 172.21.15.71:0/3111336459 shutdown_connections 2024-02-20T12:52:11.855 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:11.853+0000 7f50b4db9700 1 -- 172.21.15.71:0/3111336459 wait complete. 2024-02-20T12:52:11.855 INFO:teuthology.orchestra.run.smithi071.stderr:dumped monmap epoch 1 2024-02-20T12:52:12.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:52:11 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:52:11.852477+0000 mon.smithi064 (mon.0) 120 : audit [DBG] from='client.? 172.21.15.71:0/3111336459' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-20T12:52:13.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:52:12 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:52:11.713858+0000 mgr.smithi064.vriyor (mgr.14156) 41 : cluster [DBG] pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:52:13.499 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-20T12:52:13.499 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/daemon-base:latest-pacific shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 680ee126-cfee-11ee-95bb-87774f69a715 -- ceph mon dump -f json 2024-02-20T12:52:15.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:52:14 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:52:13.714216+0000 mgr.smithi064.vriyor (mgr.14156) 42 : cluster [DBG] pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:52:15.505 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:15.502+0000 7fab66d48700 1 -- 172.21.15.71:0/1551802407 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fab600f6590 msgr2=0x7fab600f69b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:52:15.505 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:15.502+0000 7fab66d48700 1 --2- 172.21.15.71:0/1551802407 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fab600f6590 0x7fab600f69b0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 rx=0x7fab54004760 tx=0x7fab54009ad0).stop 2024-02-20T12:52:15.505 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:15.502+0000 7fab66d48700 1 -- 172.21.15.71:0/1551802407 shutdown_connections 2024-02-20T12:52:15.505 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:15.502+0000 7fab66d48700 1 --2- 172.21.15.71:0/1551802407 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fab600f6590 0x7fab600f69b0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:52:15.505 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:15.502+0000 7fab66d48700 1 -- 172.21.15.71:0/1551802407 >> 172.21.15.71:0/1551802407 conn(0x7fab600f1cd0 msgr2=0x7fab600f4110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:52:15.506 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:15.502+0000 7fab66d48700 1 -- 172.21.15.71:0/1551802407 shutdown_connections 2024-02-20T12:52:15.506 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:15.502+0000 7fab66d48700 1 -- 172.21.15.71:0/1551802407 wait complete. 2024-02-20T12:52:15.506 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:15.502+0000 7fab66d48700 1 Processor -- start 2024-02-20T12:52:15.506 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:15.502+0000 7fab66d48700 1 -- start start 2024-02-20T12:52:15.506 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:15.503+0000 7fab66d48700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fab600f6590 0x7fab601067c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:52:15.506 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:15.503+0000 7fab66d48700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab60106cb0 con 0x7fab600f6590 2024-02-20T12:52:15.506 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:15.503+0000 7fab64ae4700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fab600f6590 0x7fab601067c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:52:15.506 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:15.503+0000 7fab64ae4700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fab600f6590 0x7fab601067c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.71:48736/0 (socket says 172.21.15.71:48736) 2024-02-20T12:52:15.506 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:15.503+0000 7fab64ae4700 1 -- 172.21.15.71:0/409377287 learned_addr learned my addr 172.21.15.71:0/409377287 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-02-20T12:52:15.506 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:15.503+0000 7fab64ae4700 1 -- 172.21.15.71:0/409377287 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab54009d70 con 0x7fab600f6590 2024-02-20T12:52:15.506 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:15.503+0000 7fab64ae4700 1 --2- 172.21.15.71:0/409377287 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fab600f6590 0x7fab601067c0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 rx=0x7fab54004170 tx=0x7fab54009ad0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:52:15.506 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:15.504+0000 7fab5d7fa700 1 -- 172.21.15.71:0/409377287 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fab5400e040 con 0x7fab600f6590 2024-02-20T12:52:15.506 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:15.504+0000 7fab66d48700 1 -- 172.21.15.71:0/409377287 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab60106eb0 con 0x7fab600f6590 2024-02-20T12:52:15.506 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:15.504+0000 7fab5d7fa700 1 -- 172.21.15.71:0/409377287 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(25 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7fab54008ef0 con 0x7fab600f6590 2024-02-20T12:52:15.507 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:15.504+0000 7fab5d7fa700 1 -- 172.21.15.71:0/409377287 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fab54014c90 con 0x7fab600f6590 2024-02-20T12:52:15.507 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:15.504+0000 7fab66d48700 1 -- 172.21.15.71:0/409377287 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab60107340 con 0x7fab600f6590 2024-02-20T12:52:15.507 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:15.505+0000 7fab5d7fa700 1 -- 172.21.15.71:0/409377287 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 37151+0+0 (secure 0 0 0) 0x7fab54014420 con 0x7fab600f6590 2024-02-20T12:52:15.507 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:15.505+0000 7fab5d7fa700 1 --2- 172.21.15.71:0/409377287 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fab50031090 0x7fab50033550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:52:15.508 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:15.505+0000 7fab66d48700 1 -- 172.21.15.71:0/409377287 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab60045bb0 con 0x7fab600f6590 2024-02-20T12:52:15.508 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:15.505+0000 7fab5d7fa700 1 -- 172.21.15.71:0/409377287 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1065+0+0 (secure 0 0 0) 0x7fab540421f0 con 0x7fab600f6590 2024-02-20T12:52:15.508 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:15.505+0000 7fab5ffff700 1 --2- 172.21.15.71:0/409377287 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fab50031090 0x7fab50033550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:52:15.508 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:15.505+0000 7fab5ffff700 1 --2- 172.21.15.71:0/409377287 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fab50031090 0x7fab50033550 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 rx=0x7fab4c0096a0 tx=0x7fab4c006b40).ready entity=mgr.14156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:52:15.511 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:15.510+0000 7fab5d7fa700 1 -- 172.21.15.71:0/409377287 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+153237 (secure 0 0 0) 0x7fab54026300 con 0x7fab600f6590 2024-02-20T12:52:15.719 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:15.717+0000 7fab66d48700 1 -- 172.21.15.71:0/409377287 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fab601081f0 con 0x7fab600f6590 2024-02-20T12:52:15.719 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:15.718+0000 7fab5d7fa700 1 -- 172.21.15.71:0/409377287 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7fab54012b20 con 0x7fab600f6590 2024-02-20T12:52:15.720 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-20T12:52:15.720 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":1,"fsid":"680ee126-cfee-11ee-95bb-87774f69a715","modified":"2024-02-20T12:49:44.699059Z","created":"2024-02-20T12:49:44.699059Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi064","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:3300","nonce":0},{"type":"v1","addr":"172.21.15.64:6789","nonce":0}]},"addr":"172.21.15.64:6789/0","public_addr":"172.21.15.64:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-20T12:52:15.721 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:15.720+0000 7fab66d48700 1 -- 172.21.15.71:0/409377287 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fab50031090 msgr2=0x7fab50033550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:52:15.722 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:15.720+0000 7fab66d48700 1 --2- 172.21.15.71:0/409377287 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fab50031090 0x7fab50033550 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 rx=0x7fab4c0096a0 tx=0x7fab4c006b40).stop 2024-02-20T12:52:15.722 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:15.720+0000 7fab66d48700 1 -- 172.21.15.71:0/409377287 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fab600f6590 msgr2=0x7fab601067c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:52:15.722 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:15.720+0000 7fab66d48700 1 --2- 172.21.15.71:0/409377287 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fab600f6590 0x7fab601067c0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 rx=0x7fab54004170 tx=0x7fab54009ad0).stop 2024-02-20T12:52:15.722 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:15.720+0000 7fab66d48700 1 -- 172.21.15.71:0/409377287 shutdown_connections 2024-02-20T12:52:15.722 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:15.720+0000 7fab66d48700 1 --2- 172.21.15.71:0/409377287 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fab50031090 0x7fab50033550 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:52:15.722 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:15.720+0000 7fab66d48700 1 --2- 172.21.15.71:0/409377287 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fab600f6590 0x7fab601067c0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:52:15.722 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:15.720+0000 7fab66d48700 1 -- 172.21.15.71:0/409377287 >> 172.21.15.71:0/409377287 conn(0x7fab600f1cd0 msgr2=0x7fab600f27c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:52:15.722 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:15.720+0000 7fab66d48700 1 -- 172.21.15.71:0/409377287 shutdown_connections 2024-02-20T12:52:15.722 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:15.720+0000 7fab66d48700 1 -- 172.21.15.71:0/409377287 wait complete. 2024-02-20T12:52:15.722 INFO:teuthology.orchestra.run.smithi071.stderr:dumped monmap epoch 1 2024-02-20T12:52:16.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:52:15 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:52:15.719790+0000 mon.smithi064 (mon.0) 121 : audit [DBG] from='client.? 172.21.15.71:0/409377287' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-20T12:52:17.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:52:16 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:52:15.714575+0000 mgr.smithi064.vriyor (mgr.14156) 43 : cluster [DBG] pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:52:17.353 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-20T12:52:17.354 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/daemon-base:latest-pacific shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 680ee126-cfee-11ee-95bb-87774f69a715 -- ceph mon dump -f json 2024-02-20T12:52:19.165 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:19.154+0000 7fc9c3143700 1 -- 172.21.15.71:0/3163915455 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc9bc0602c0 msgr2=0x7fc9bc0f9200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:52:19.166 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:19.154+0000 7fc9c3143700 1 --2- 172.21.15.71:0/3163915455 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc9bc0602c0 0x7fc9bc0f9200 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 rx=0x7fc9b0009fb0 tx=0x7fc9b0009b30).stop 2024-02-20T12:52:19.166 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:19.154+0000 7fc9c3143700 1 -- 172.21.15.71:0/3163915455 shutdown_connections 2024-02-20T12:52:19.166 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:19.154+0000 7fc9c3143700 1 --2- 172.21.15.71:0/3163915455 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc9bc0602c0 0x7fc9bc0f9200 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:52:19.166 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:19.154+0000 7fc9c3143700 1 -- 172.21.15.71:0/3163915455 >> 172.21.15.71:0/3163915455 conn(0x7fc9bc0f0be0 msgr2=0x7fc9bc0f3000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:52:19.166 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:19.154+0000 7fc9c3143700 1 -- 172.21.15.71:0/3163915455 shutdown_connections 2024-02-20T12:52:19.166 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:19.154+0000 7fc9c3143700 1 -- 172.21.15.71:0/3163915455 wait complete. 2024-02-20T12:52:19.166 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:19.154+0000 7fc9c3143700 1 Processor -- start 2024-02-20T12:52:19.166 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:19.155+0000 7fc9c3143700 1 -- start start 2024-02-20T12:52:19.166 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:19.155+0000 7fc9c3143700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc9bc0602c0 0x7fc9bc106730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:52:19.166 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:19.155+0000 7fc9c3143700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9bc106c20 con 0x7fc9bc0602c0 2024-02-20T12:52:19.166 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:19.155+0000 7fc9c0edf700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc9bc0602c0 0x7fc9bc106730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:52:19.167 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:19.155+0000 7fc9c0edf700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc9bc0602c0 0x7fc9bc106730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.71:40032/0 (socket says 172.21.15.71:40032) 2024-02-20T12:52:19.167 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:19.155+0000 7fc9c0edf700 1 -- 172.21.15.71:0/1515814832 learned_addr learned my addr 172.21.15.71:0/1515814832 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-02-20T12:52:19.167 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:19.155+0000 7fc9c0edf700 1 -- 172.21.15.71:0/1515814832 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9b0005040 con 0x7fc9bc0602c0 2024-02-20T12:52:19.167 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:19.155+0000 7fc9c0edf700 1 --2- 172.21.15.71:0/1515814832 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc9bc0602c0 0x7fc9bc106730 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 rx=0x7fc9b0013fd0 tx=0x7fc9b0009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:52:19.167 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:19.156+0000 7fc9b97fa700 1 -- 172.21.15.71:0/1515814832 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc9b0018070 con 0x7fc9bc0602c0 2024-02-20T12:52:19.167 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:19.156+0000 7fc9b97fa700 1 -- 172.21.15.71:0/1515814832 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(25 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7fc9b0007b10 con 0x7fc9bc0602c0 2024-02-20T12:52:19.167 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:19.156+0000 7fc9c3143700 1 -- 172.21.15.71:0/1515814832 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9bc106e20 con 0x7fc9bc0602c0 2024-02-20T12:52:19.167 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:19.156+0000 7fc9b97fa700 1 -- 172.21.15.71:0/1515814832 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc9b001c3c0 con 0x7fc9bc0602c0 2024-02-20T12:52:19.167 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:19.156+0000 7fc9c3143700 1 -- 172.21.15.71:0/1515814832 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9bc1072b0 con 0x7fc9bc0602c0 2024-02-20T12:52:19.167 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:19.156+0000 7fc9b97fa700 1 -- 172.21.15.71:0/1515814832 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 37151+0+0 (secure 0 0 0) 0x7fc9b0007c80 con 0x7fc9bc0602c0 2024-02-20T12:52:19.167 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:19.157+0000 7fc9c3143700 1 -- 172.21.15.71:0/1515814832 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9bc045bb0 con 0x7fc9bc0602c0 2024-02-20T12:52:19.167 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:19.157+0000 7fc9b97fa700 1 --2- 172.21.15.71:0/1515814832 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fc9ac031090 0x7fc9ac033550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:52:19.167 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:19.157+0000 7fc9b97fa700 1 -- 172.21.15.71:0/1515814832 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1065+0+0 (secure 0 0 0) 0x7fc9b0042a50 con 0x7fc9bc0602c0 2024-02-20T12:52:19.167 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:19.157+0000 7fc9bbfff700 1 --2- 172.21.15.71:0/1515814832 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fc9ac031090 0x7fc9ac033550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:52:19.168 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:19.157+0000 7fc9bbfff700 1 --2- 172.21.15.71:0/1515814832 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fc9ac031090 0x7fc9ac033550 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 rx=0x7fc9a80096a0 tx=0x7fc9a8006b40).ready entity=mgr.14156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:52:19.168 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:19.162+0000 7fc9b97fa700 1 -- 172.21.15.71:0/1515814832 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+153237 (secure 0 0 0) 0x7fc9b0025b40 con 0x7fc9bc0602c0 2024-02-20T12:52:19.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:52:18 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:52:17.714841+0000 mgr.smithi064.vriyor (mgr.14156) 44 : cluster [DBG] pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:52:19.368 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:19.368+0000 7fc9c3143700 1 -- 172.21.15.71:0/1515814832 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fc9bc108160 con 0x7fc9bc0602c0 2024-02-20T12:52:19.369 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:19.368+0000 7fc9b97fa700 1 -- 172.21.15.71:0/1515814832 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7fc9b0025b40 con 0x7fc9bc0602c0 2024-02-20T12:52:19.369 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-20T12:52:19.369 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":1,"fsid":"680ee126-cfee-11ee-95bb-87774f69a715","modified":"2024-02-20T12:49:44.699059Z","created":"2024-02-20T12:49:44.699059Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi064","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:3300","nonce":0},{"type":"v1","addr":"172.21.15.64:6789","nonce":0}]},"addr":"172.21.15.64:6789/0","public_addr":"172.21.15.64:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-20T12:52:19.370 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:19.370+0000 7fc9c3143700 1 -- 172.21.15.71:0/1515814832 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fc9ac031090 msgr2=0x7fc9ac033550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:52:19.371 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:19.370+0000 7fc9c3143700 1 --2- 172.21.15.71:0/1515814832 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fc9ac031090 0x7fc9ac033550 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 rx=0x7fc9a80096a0 tx=0x7fc9a8006b40).stop 2024-02-20T12:52:19.371 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:19.370+0000 7fc9c3143700 1 -- 172.21.15.71:0/1515814832 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc9bc0602c0 msgr2=0x7fc9bc106730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:52:19.371 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:19.370+0000 7fc9c3143700 1 --2- 172.21.15.71:0/1515814832 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc9bc0602c0 0x7fc9bc106730 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 rx=0x7fc9b0013fd0 tx=0x7fc9b0009b30).stop 2024-02-20T12:52:19.371 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:19.370+0000 7fc9c3143700 1 -- 172.21.15.71:0/1515814832 shutdown_connections 2024-02-20T12:52:19.371 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:19.370+0000 7fc9c3143700 1 --2- 172.21.15.71:0/1515814832 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fc9ac031090 0x7fc9ac033550 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:52:19.371 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:19.370+0000 7fc9c3143700 1 --2- 172.21.15.71:0/1515814832 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc9bc0602c0 0x7fc9bc106730 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:52:19.371 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:19.370+0000 7fc9c3143700 1 -- 172.21.15.71:0/1515814832 >> 172.21.15.71:0/1515814832 conn(0x7fc9bc0f0be0 msgr2=0x7fc9bc0f1630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:52:19.371 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:19.370+0000 7fc9c3143700 1 -- 172.21.15.71:0/1515814832 shutdown_connections 2024-02-20T12:52:19.371 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:19.370+0000 7fc9c3143700 1 -- 172.21.15.71:0/1515814832 wait complete. 2024-02-20T12:52:19.371 INFO:teuthology.orchestra.run.smithi071.stderr:dumped monmap epoch 1 2024-02-20T12:52:20.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:52:19 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:52:19.369204+0000 mon.smithi064 (mon.0) 122 : audit [DBG] from='client.? 172.21.15.71:0/1515814832' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-20T12:52:21.049 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-20T12:52:21.049 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/daemon-base:latest-pacific shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 680ee126-cfee-11ee-95bb-87774f69a715 -- ceph mon dump -f json 2024-02-20T12:52:21.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:52:20 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:52:19.715216+0000 mgr.smithi064.vriyor (mgr.14156) 45 : cluster [DBG] pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:52:22.854 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:22.852+0000 7f91d4571700 1 -- 172.21.15.71:0/1054237593 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f91cc0f6590 msgr2=0x7f91cc0f69b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:52:22.854 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:22.852+0000 7f91d4571700 1 --2- 172.21.15.71:0/1054237593 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f91cc0f6590 0x7f91cc0f69b0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 rx=0x7f91c8004760 tx=0x7f91c8009ad0).stop 2024-02-20T12:52:22.854 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:22.852+0000 7f91d4571700 1 -- 172.21.15.71:0/1054237593 shutdown_connections 2024-02-20T12:52:22.854 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:22.852+0000 7f91d4571700 1 --2- 172.21.15.71:0/1054237593 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f91cc0f6590 0x7f91cc0f69b0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:52:22.854 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:22.852+0000 7f91d4571700 1 -- 172.21.15.71:0/1054237593 >> 172.21.15.71:0/1054237593 conn(0x7f91cc0f1cd0 msgr2=0x7f91cc0f4110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:52:22.854 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:22.852+0000 7f91d4571700 1 -- 172.21.15.71:0/1054237593 shutdown_connections 2024-02-20T12:52:22.854 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:22.852+0000 7f91d4571700 1 -- 172.21.15.71:0/1054237593 wait complete. 2024-02-20T12:52:22.854 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:22.852+0000 7f91d4571700 1 Processor -- start 2024-02-20T12:52:22.854 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:22.852+0000 7f91d4571700 1 -- start start 2024-02-20T12:52:22.855 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:22.853+0000 7f91d4571700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f91cc0f6590 0x7f91cc1067c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:52:22.855 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:22.853+0000 7f91d4571700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91cc106cb0 con 0x7f91cc0f6590 2024-02-20T12:52:22.855 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:22.853+0000 7f91d230d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f91cc0f6590 0x7f91cc1067c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:52:22.855 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:22.853+0000 7f91d230d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f91cc0f6590 0x7f91cc1067c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.71:40042/0 (socket says 172.21.15.71:40042) 2024-02-20T12:52:22.855 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:22.853+0000 7f91d230d700 1 -- 172.21.15.71:0/753454722 learned_addr learned my addr 172.21.15.71:0/753454722 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-02-20T12:52:22.855 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:22.853+0000 7f91d230d700 1 -- 172.21.15.71:0/753454722 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91c8009d70 con 0x7f91cc0f6590 2024-02-20T12:52:22.855 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:22.853+0000 7f91d230d700 1 --2- 172.21.15.71:0/753454722 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f91cc0f6590 0x7f91cc1067c0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 rx=0x7f91c8004170 tx=0x7f91c8009ad0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:52:22.855 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:22.853+0000 7f91c2ffd700 1 -- 172.21.15.71:0/753454722 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f91c800e040 con 0x7f91cc0f6590 2024-02-20T12:52:22.855 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:22.853+0000 7f91d4571700 1 -- 172.21.15.71:0/753454722 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91cc106eb0 con 0x7f91cc0f6590 2024-02-20T12:52:22.855 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:22.853+0000 7f91c2ffd700 1 -- 172.21.15.71:0/753454722 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(25 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f91c8008ef0 con 0x7f91cc0f6590 2024-02-20T12:52:22.855 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:22.853+0000 7f91c2ffd700 1 -- 172.21.15.71:0/753454722 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f91c8014c90 con 0x7f91cc0f6590 2024-02-20T12:52:22.855 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:22.853+0000 7f91d4571700 1 -- 172.21.15.71:0/753454722 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91cc107340 con 0x7f91cc0f6590 2024-02-20T12:52:22.856 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:22.854+0000 7f91c2ffd700 1 -- 172.21.15.71:0/753454722 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 37151+0+0 (secure 0 0 0) 0x7f91c8014420 con 0x7f91cc0f6590 2024-02-20T12:52:22.856 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:22.854+0000 7f91c2ffd700 1 --2- 172.21.15.71:0/753454722 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f91b8031090 0x7f91b8033550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:52:22.856 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:22.854+0000 7f91c2ffd700 1 -- 172.21.15.71:0/753454722 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1065+0+0 (secure 0 0 0) 0x7f91c80421f0 con 0x7f91cc0f6590 2024-02-20T12:52:22.857 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:22.854+0000 7f91d4571700 1 -- 172.21.15.71:0/753454722 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91cc045bb0 con 0x7f91cc0f6590 2024-02-20T12:52:22.857 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:22.854+0000 7f91d1b0c700 1 --2- 172.21.15.71:0/753454722 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f91b8031090 0x7f91b8033550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:52:22.857 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:22.855+0000 7f91d1b0c700 1 --2- 172.21.15.71:0/753454722 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f91b8031090 0x7f91b8033550 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 rx=0x7f91bc0096a0 tx=0x7f91bc006b40).ready entity=mgr.14156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:52:22.859 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:22.859+0000 7f91c2ffd700 1 -- 172.21.15.71:0/753454722 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+153237 (secure 0 0 0) 0x7f91c8026300 con 0x7f91cc0f6590 2024-02-20T12:52:23.064 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:23.064+0000 7f91d4571700 1 -- 172.21.15.71:0/753454722 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f91cc0fc170 con 0x7f91cc0f6590 2024-02-20T12:52:23.065 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:23.064+0000 7f91c2ffd700 1 -- 172.21.15.71:0/753454722 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7f91c8012b20 con 0x7f91cc0f6590 2024-02-20T12:52:23.065 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-20T12:52:23.065 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":1,"fsid":"680ee126-cfee-11ee-95bb-87774f69a715","modified":"2024-02-20T12:49:44.699059Z","created":"2024-02-20T12:49:44.699059Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi064","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:3300","nonce":0},{"type":"v1","addr":"172.21.15.64:6789","nonce":0}]},"addr":"172.21.15.64:6789/0","public_addr":"172.21.15.64:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-20T12:52:23.067 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:23.066+0000 7f91d4571700 1 -- 172.21.15.71:0/753454722 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f91b8031090 msgr2=0x7f91b8033550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:52:23.067 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:23.066+0000 7f91d4571700 1 --2- 172.21.15.71:0/753454722 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f91b8031090 0x7f91b8033550 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 rx=0x7f91bc0096a0 tx=0x7f91bc006b40).stop 2024-02-20T12:52:23.067 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:23.066+0000 7f91d4571700 1 -- 172.21.15.71:0/753454722 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f91cc0f6590 msgr2=0x7f91cc1067c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:52:23.067 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:23.066+0000 7f91d4571700 1 --2- 172.21.15.71:0/753454722 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f91cc0f6590 0x7f91cc1067c0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 rx=0x7f91c8004170 tx=0x7f91c8009ad0).stop 2024-02-20T12:52:23.067 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:23.066+0000 7f91d4571700 1 -- 172.21.15.71:0/753454722 shutdown_connections 2024-02-20T12:52:23.067 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:23.066+0000 7f91d4571700 1 --2- 172.21.15.71:0/753454722 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f91b8031090 0x7f91b8033550 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:52:23.067 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:23.066+0000 7f91d4571700 1 --2- 172.21.15.71:0/753454722 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f91cc0f6590 0x7f91cc1067c0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:52:23.067 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:23.066+0000 7f91d4571700 1 -- 172.21.15.71:0/753454722 >> 172.21.15.71:0/753454722 conn(0x7f91cc0f1cd0 msgr2=0x7f91cc0f27c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:52:23.067 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:23.066+0000 7f91d4571700 1 -- 172.21.15.71:0/753454722 shutdown_connections 2024-02-20T12:52:23.067 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:23.066+0000 7f91d4571700 1 -- 172.21.15.71:0/753454722 wait complete. 2024-02-20T12:52:23.068 INFO:teuthology.orchestra.run.smithi071.stderr:dumped monmap epoch 1 2024-02-20T12:52:23.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:52:22 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:52:21.715556+0000 mgr.smithi064.vriyor (mgr.14156) 46 : cluster [DBG] pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:52:24.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:52:23 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:52:23.065319+0000 mon.smithi064 (mon.0) 123 : audit [DBG] from='client.? 172.21.15.71:0/753454722' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-20T12:52:24.670 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-20T12:52:24.670 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/daemon-base:latest-pacific shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 680ee126-cfee-11ee-95bb-87774f69a715 -- ceph mon dump -f json 2024-02-20T12:52:25.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:52:24 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:52:23.715938+0000 mgr.smithi064.vriyor (mgr.14156) 47 : cluster [DBG] pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:52:26.687 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:26.685+0000 7f721f59e700 1 -- 172.21.15.71:0/4105020147 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f72200f52d0 msgr2=0x7f72200f56f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:52:26.687 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:26.685+0000 7f721f59e700 1 --2- 172.21.15.71:0/4105020147 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f72200f52d0 0x7f72200f56f0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 rx=0x7f7210009fb0 tx=0x7f7210009b30).stop 2024-02-20T12:52:26.687 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:26.685+0000 7f721f59e700 1 -- 172.21.15.71:0/4105020147 shutdown_connections 2024-02-20T12:52:26.687 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:26.685+0000 7f721f59e700 1 --2- 172.21.15.71:0/4105020147 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f72200f52d0 0x7f72200f56f0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:52:26.687 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:26.685+0000 7f721f59e700 1 -- 172.21.15.71:0/4105020147 >> 172.21.15.71:0/4105020147 conn(0x7f72200f0a30 msgr2=0x7f72200f2e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:52:26.687 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:26.685+0000 7f721f59e700 1 -- 172.21.15.71:0/4105020147 shutdown_connections 2024-02-20T12:52:26.687 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:26.685+0000 7f721f59e700 1 -- 172.21.15.71:0/4105020147 wait complete. 2024-02-20T12:52:26.688 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:26.685+0000 7f721f59e700 1 Processor -- start 2024-02-20T12:52:26.688 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:26.685+0000 7f721f59e700 1 -- start start 2024-02-20T12:52:26.688 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:26.686+0000 7f721f59e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f72200f52d0 0x7f7220109390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:52:26.688 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:26.686+0000 7f721f59e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7220060300 con 0x7f72200f52d0 2024-02-20T12:52:26.688 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:26.686+0000 7f721e59c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f72200f52d0 0x7f7220109390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:52:26.688 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:26.686+0000 7f721e59c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f72200f52d0 0x7f7220109390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.71:40058/0 (socket says 172.21.15.71:40058) 2024-02-20T12:52:26.688 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:26.686+0000 7f721e59c700 1 -- 172.21.15.71:0/1650591104 learned_addr learned my addr 172.21.15.71:0/1650591104 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-02-20T12:52:26.688 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:26.686+0000 7f721e59c700 1 -- 172.21.15.71:0/1650591104 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7210005040 con 0x7f72200f52d0 2024-02-20T12:52:26.688 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:26.686+0000 7f721e59c700 1 --2- 172.21.15.71:0/1650591104 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f72200f52d0 0x7f7220109390 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 rx=0x7f72100089f0 tx=0x7f721000b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:52:26.688 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:26.686+0000 7f720effd700 1 -- 172.21.15.71:0/1650591104 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7210009320 con 0x7f72200f52d0 2024-02-20T12:52:26.688 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:26.686+0000 7f721f59e700 1 -- 172.21.15.71:0/1650591104 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7220060500 con 0x7f72200f52d0 2024-02-20T12:52:26.688 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:26.686+0000 7f720effd700 1 -- 172.21.15.71:0/1650591104 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(25 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f7210013ec0 con 0x7f72200f52d0 2024-02-20T12:52:26.689 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:26.686+0000 7f720effd700 1 -- 172.21.15.71:0/1650591104 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7210014d30 con 0x7f72200f52d0 2024-02-20T12:52:26.689 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:26.686+0000 7f721f59e700 1 -- 172.21.15.71:0/1650591104 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7220109a40 con 0x7f72200f52d0 2024-02-20T12:52:26.690 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:26.687+0000 7f720effd700 1 -- 172.21.15.71:0/1650591104 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 37151+0+0 (secure 0 0 0) 0x7f72100095f0 con 0x7f72200f52d0 2024-02-20T12:52:26.690 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:26.687+0000 7f721f59e700 1 -- 172.21.15.71:0/1650591104 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72200591e0 con 0x7f72200f52d0 2024-02-20T12:52:26.690 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:26.687+0000 7f720effd700 1 --2- 172.21.15.71:0/1650591104 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f7208031090 0x7f7208033550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:52:26.690 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:26.687+0000 7f720effd700 1 -- 172.21.15.71:0/1650591104 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1065+0+0 (secure 0 0 0) 0x7f72100424a0 con 0x7f72200f52d0 2024-02-20T12:52:26.690 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:26.688+0000 7f721dd9b700 1 --2- 172.21.15.71:0/1650591104 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f7208031090 0x7f7208033550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:52:26.690 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:26.688+0000 7f721dd9b700 1 --2- 172.21.15.71:0/1650591104 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f7208031090 0x7f7208033550 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 rx=0x7f72140096a0 tx=0x7f7214006b40).ready entity=mgr.14156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:52:26.692 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:26.692+0000 7f720effd700 1 -- 172.21.15.71:0/1650591104 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+153237 (secure 0 0 0) 0x7f721001d7f0 con 0x7f72200f52d0 2024-02-20T12:52:26.903 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:26.903+0000 7f721f59e700 1 -- 172.21.15.71:0/1650591104 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f7220045bb0 con 0x7f72200f52d0 2024-02-20T12:52:26.904 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:26.903+0000 7f720effd700 1 -- 172.21.15.71:0/1650591104 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7f721001d7f0 con 0x7f72200f52d0 2024-02-20T12:52:26.904 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-20T12:52:26.904 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":1,"fsid":"680ee126-cfee-11ee-95bb-87774f69a715","modified":"2024-02-20T12:49:44.699059Z","created":"2024-02-20T12:49:44.699059Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi064","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:3300","nonce":0},{"type":"v1","addr":"172.21.15.64:6789","nonce":0}]},"addr":"172.21.15.64:6789/0","public_addr":"172.21.15.64:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-20T12:52:26.905 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:26.905+0000 7f721f59e700 1 -- 172.21.15.71:0/1650591104 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f7208031090 msgr2=0x7f7208033550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:52:26.906 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:26.905+0000 7f721f59e700 1 --2- 172.21.15.71:0/1650591104 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f7208031090 0x7f7208033550 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 rx=0x7f72140096a0 tx=0x7f7214006b40).stop 2024-02-20T12:52:26.906 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:26.905+0000 7f721f59e700 1 -- 172.21.15.71:0/1650591104 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f72200f52d0 msgr2=0x7f7220109390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:52:26.906 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:26.905+0000 7f721f59e700 1 --2- 172.21.15.71:0/1650591104 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f72200f52d0 0x7f7220109390 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 rx=0x7f72100089f0 tx=0x7f721000b040).stop 2024-02-20T12:52:26.906 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:26.905+0000 7f721f59e700 1 -- 172.21.15.71:0/1650591104 shutdown_connections 2024-02-20T12:52:26.906 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:26.905+0000 7f721f59e700 1 --2- 172.21.15.71:0/1650591104 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f7208031090 0x7f7208033550 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:52:26.906 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:26.905+0000 7f721f59e700 1 --2- 172.21.15.71:0/1650591104 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f72200f52d0 0x7f7220109390 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:52:26.906 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:26.905+0000 7f721f59e700 1 -- 172.21.15.71:0/1650591104 >> 172.21.15.71:0/1650591104 conn(0x7f72200f0a30 msgr2=0x7f72200f1560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:52:26.906 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:26.905+0000 7f721f59e700 1 -- 172.21.15.71:0/1650591104 shutdown_connections 2024-02-20T12:52:26.906 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:26.905+0000 7f721f59e700 1 -- 172.21.15.71:0/1650591104 wait complete. 2024-02-20T12:52:26.907 INFO:teuthology.orchestra.run.smithi071.stderr:dumped monmap epoch 1 2024-02-20T12:52:27.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:52:26 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:52:25.716324+0000 mgr.smithi064.vriyor (mgr.14156) 48 : cluster [DBG] pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:52:27.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:52:26 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:52:26.904145+0000 mon.smithi064 (mon.0) 124 : audit [DBG] from='client.? 172.21.15.71:0/1650591104' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-20T12:52:28.465 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-20T12:52:28.466 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/daemon-base:latest-pacific shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 680ee126-cfee-11ee-95bb-87774f69a715 -- ceph mon dump -f json 2024-02-20T12:52:29.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:52:28 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:52:27.716638+0000 mgr.smithi064.vriyor (mgr.14156) 49 : cluster [DBG] pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:52:30.436 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:30.400+0000 7fb23bb6a700 1 -- 172.21.15.71:0/3433302942 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb2340f5480 msgr2=0x7fb2340f58a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:52:30.436 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:30.400+0000 7fb23bb6a700 1 --2- 172.21.15.71:0/3433302942 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb2340f5480 0x7fb2340f58a0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 rx=0x7fb228009fb0 tx=0x7fb228009b30).stop 2024-02-20T12:52:30.436 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:30.400+0000 7fb23bb6a700 1 -- 172.21.15.71:0/3433302942 shutdown_connections 2024-02-20T12:52:30.437 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:30.400+0000 7fb23bb6a700 1 --2- 172.21.15.71:0/3433302942 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb2340f5480 0x7fb2340f58a0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:52:30.437 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:30.400+0000 7fb23bb6a700 1 -- 172.21.15.71:0/3433302942 >> 172.21.15.71:0/3433302942 conn(0x7fb2340f0be0 msgr2=0x7fb2340f3000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:52:30.437 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:30.401+0000 7fb23bb6a700 1 -- 172.21.15.71:0/3433302942 shutdown_connections 2024-02-20T12:52:30.437 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:30.401+0000 7fb23bb6a700 1 -- 172.21.15.71:0/3433302942 wait complete. 2024-02-20T12:52:30.437 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:30.401+0000 7fb23bb6a700 1 Processor -- start 2024-02-20T12:52:30.437 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:30.401+0000 7fb23bb6a700 1 -- start start 2024-02-20T12:52:30.437 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:30.401+0000 7fb23bb6a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb2340f5480 0x7fb2341094f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:52:30.437 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:30.401+0000 7fb23bb6a700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb234060320 con 0x7fb2340f5480 2024-02-20T12:52:30.437 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:30.402+0000 7fb239906700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb2340f5480 0x7fb2341094f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:52:30.437 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:30.402+0000 7fb239906700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb2340f5480 0x7fb2341094f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.71:45562/0 (socket says 172.21.15.71:45562) 2024-02-20T12:52:30.437 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:30.402+0000 7fb239906700 1 -- 172.21.15.71:0/797590329 learned_addr learned my addr 172.21.15.71:0/797590329 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-02-20T12:52:30.437 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:30.402+0000 7fb239906700 1 -- 172.21.15.71:0/797590329 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb228005040 con 0x7fb2340f5480 2024-02-20T12:52:30.437 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:30.402+0000 7fb239906700 1 --2- 172.21.15.71:0/797590329 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb2340f5480 0x7fb2341094f0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 rx=0x7fb2280089f0 tx=0x7fb22800b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:52:30.437 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:30.402+0000 7fb2267fc700 1 -- 172.21.15.71:0/797590329 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb228009320 con 0x7fb2340f5480 2024-02-20T12:52:30.438 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:30.402+0000 7fb23bb6a700 1 -- 172.21.15.71:0/797590329 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb234060520 con 0x7fb2340f5480 2024-02-20T12:52:30.438 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:30.402+0000 7fb2267fc700 1 -- 172.21.15.71:0/797590329 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(25 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7fb228013ec0 con 0x7fb2340f5480 2024-02-20T12:52:30.438 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:30.402+0000 7fb2267fc700 1 -- 172.21.15.71:0/797590329 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb228014d30 con 0x7fb2340f5480 2024-02-20T12:52:30.438 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:30.402+0000 7fb23bb6a700 1 -- 172.21.15.71:0/797590329 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb234109ba0 con 0x7fb2340f5480 2024-02-20T12:52:30.438 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:30.403+0000 7fb2267fc700 1 -- 172.21.15.71:0/797590329 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 37151+0+0 (secure 0 0 0) 0x7fb2280095f0 con 0x7fb2340f5480 2024-02-20T12:52:30.438 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:30.403+0000 7fb2267fc700 1 --2- 172.21.15.71:0/797590329 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fb220031090 0x7fb220033550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:52:30.438 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:30.403+0000 7fb2267fc700 1 -- 172.21.15.71:0/797590329 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1065+0+0 (secure 0 0 0) 0x7fb2280424a0 con 0x7fb2340f5480 2024-02-20T12:52:30.438 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:30.403+0000 7fb23bb6a700 1 -- 172.21.15.71:0/797590329 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb234045bb0 con 0x7fb2340f5480 2024-02-20T12:52:30.438 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:30.404+0000 7fb239105700 1 --2- 172.21.15.71:0/797590329 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fb220031090 0x7fb220033550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:52:30.438 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:30.404+0000 7fb239105700 1 --2- 172.21.15.71:0/797590329 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fb220031090 0x7fb220033550 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0x7fb2300096a0 tx=0x7fb230006b40).ready entity=mgr.14156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:52:30.438 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:30.409+0000 7fb2267fc700 1 -- 172.21.15.71:0/797590329 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+153237 (secure 0 0 0) 0x7fb22801dd70 con 0x7fb2340f5480 2024-02-20T12:52:30.616 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:30.616+0000 7fb23bb6a700 1 -- 172.21.15.71:0/797590329 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fb2340fb060 con 0x7fb2340f5480 2024-02-20T12:52:30.617 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:30.616+0000 7fb2267fc700 1 -- 172.21.15.71:0/797590329 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7fb22802b020 con 0x7fb2340f5480 2024-02-20T12:52:30.617 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-20T12:52:30.617 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":1,"fsid":"680ee126-cfee-11ee-95bb-87774f69a715","modified":"2024-02-20T12:49:44.699059Z","created":"2024-02-20T12:49:44.699059Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi064","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:3300","nonce":0},{"type":"v1","addr":"172.21.15.64:6789","nonce":0}]},"addr":"172.21.15.64:6789/0","public_addr":"172.21.15.64:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-20T12:52:30.618 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:30.618+0000 7fb23bb6a700 1 -- 172.21.15.71:0/797590329 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fb220031090 msgr2=0x7fb220033550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:52:30.619 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:30.618+0000 7fb23bb6a700 1 --2- 172.21.15.71:0/797590329 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fb220031090 0x7fb220033550 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0x7fb2300096a0 tx=0x7fb230006b40).stop 2024-02-20T12:52:30.619 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:30.618+0000 7fb23bb6a700 1 -- 172.21.15.71:0/797590329 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb2340f5480 msgr2=0x7fb2341094f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:52:30.619 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:30.618+0000 7fb23bb6a700 1 --2- 172.21.15.71:0/797590329 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb2340f5480 0x7fb2341094f0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 rx=0x7fb2280089f0 tx=0x7fb22800b040).stop 2024-02-20T12:52:30.619 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:30.618+0000 7fb23bb6a700 1 -- 172.21.15.71:0/797590329 shutdown_connections 2024-02-20T12:52:30.619 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:30.618+0000 7fb23bb6a700 1 --2- 172.21.15.71:0/797590329 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fb220031090 0x7fb220033550 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:52:30.619 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:30.618+0000 7fb23bb6a700 1 --2- 172.21.15.71:0/797590329 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb2340f5480 0x7fb2341094f0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:52:30.619 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:30.618+0000 7fb23bb6a700 1 -- 172.21.15.71:0/797590329 >> 172.21.15.71:0/797590329 conn(0x7fb2340f0be0 msgr2=0x7fb2340f16d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:52:30.619 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:30.618+0000 7fb23bb6a700 1 -- 172.21.15.71:0/797590329 shutdown_connections 2024-02-20T12:52:30.619 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:30.618+0000 7fb23bb6a700 1 -- 172.21.15.71:0/797590329 wait complete. 2024-02-20T12:52:30.620 INFO:teuthology.orchestra.run.smithi071.stderr:dumped monmap epoch 1 2024-02-20T12:52:31.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:52:30 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:52:29.716952+0000 mgr.smithi064.vriyor (mgr.14156) 50 : cluster [DBG] pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:52:31.315 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:52:30 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:52:30.617206+0000 mon.smithi064 (mon.0) 125 : audit [DBG] from='client.? 172.21.15.71:0/797590329' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-20T12:52:32.307 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-20T12:52:32.307 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/daemon-base:latest-pacific shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 680ee126-cfee-11ee-95bb-87774f69a715 -- ceph mon dump -f json 2024-02-20T12:52:33.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:52:32 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:52:31.717300+0000 mgr.smithi064.vriyor (mgr.14156) 51 : cluster [DBG] pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:52:34.091 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:34.044+0000 7f3a05e42700 1 -- 172.21.15.71:0/1538557537 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3a000f5480 msgr2=0x7f3a000f58a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:52:34.091 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:34.044+0000 7f3a05e42700 1 --2- 172.21.15.71:0/1538557537 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3a000f5480 0x7f3a000f58a0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 rx=0x7f39f4009fb0 tx=0x7f39f4009b30).stop 2024-02-20T12:52:34.091 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:34.044+0000 7f3a05e42700 1 -- 172.21.15.71:0/1538557537 shutdown_connections 2024-02-20T12:52:34.091 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:34.044+0000 7f3a05e42700 1 --2- 172.21.15.71:0/1538557537 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3a000f5480 0x7f3a000f58a0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:52:34.091 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:34.044+0000 7f3a05e42700 1 -- 172.21.15.71:0/1538557537 >> 172.21.15.71:0/1538557537 conn(0x7f3a000f0be0 msgr2=0x7f3a000f3000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:52:34.091 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:34.044+0000 7f3a05e42700 1 -- 172.21.15.71:0/1538557537 shutdown_connections 2024-02-20T12:52:34.092 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:34.044+0000 7f3a05e42700 1 -- 172.21.15.71:0/1538557537 wait complete. 2024-02-20T12:52:34.092 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:34.045+0000 7f3a05e42700 1 Processor -- start 2024-02-20T12:52:34.092 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:34.045+0000 7f3a05e42700 1 -- start start 2024-02-20T12:52:34.092 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:34.045+0000 7f3a05e42700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3a000f5480 0x7f3a001067e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:52:34.092 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:34.045+0000 7f3a05e42700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a00106cd0 con 0x7f3a000f5480 2024-02-20T12:52:34.092 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:34.045+0000 7f39ff7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3a000f5480 0x7f3a001067e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:52:34.092 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:34.045+0000 7f39ff7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3a000f5480 0x7f3a001067e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.71:45580/0 (socket says 172.21.15.71:45580) 2024-02-20T12:52:34.092 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:34.045+0000 7f39ff7fe700 1 -- 172.21.15.71:0/2418929719 learned_addr learned my addr 172.21.15.71:0/2418929719 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-02-20T12:52:34.092 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:34.046+0000 7f39ff7fe700 1 -- 172.21.15.71:0/2418929719 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39f4005040 con 0x7f3a000f5480 2024-02-20T12:52:34.092 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:34.046+0000 7f39ff7fe700 1 --2- 172.21.15.71:0/2418929719 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3a000f5480 0x7f3a001067e0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 rx=0x7f39f40089f0 tx=0x7f39f400b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:52:34.092 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:34.046+0000 7f39e7fff700 1 -- 172.21.15.71:0/2418929719 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f39f4009320 con 0x7f3a000f5480 2024-02-20T12:52:34.092 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:34.046+0000 7f3a05e42700 1 -- 172.21.15.71:0/2418929719 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a00106ed0 con 0x7f3a000f5480 2024-02-20T12:52:34.092 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:34.046+0000 7f39e7fff700 1 -- 172.21.15.71:0/2418929719 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(25 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f39f4013ec0 con 0x7f3a000f5480 2024-02-20T12:52:34.092 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:34.046+0000 7f39e7fff700 1 -- 172.21.15.71:0/2418929719 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f39f4014d30 con 0x7f3a000f5480 2024-02-20T12:52:34.092 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:34.046+0000 7f3a05e42700 1 -- 172.21.15.71:0/2418929719 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a00107360 con 0x7f3a000f5480 2024-02-20T12:52:34.093 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:34.047+0000 7f39e7fff700 1 -- 172.21.15.71:0/2418929719 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 37151+0+0 (secure 0 0 0) 0x7f39f4014420 con 0x7f3a000f5480 2024-02-20T12:52:34.093 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:34.047+0000 7f39e7fff700 1 --2- 172.21.15.71:0/2418929719 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f39e8031090 0x7f39e8033550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:52:34.093 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:34.047+0000 7f3a05e42700 1 -- 172.21.15.71:0/2418929719 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a00045bb0 con 0x7f3a000f5480 2024-02-20T12:52:34.093 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:34.047+0000 7f39e7fff700 1 -- 172.21.15.71:0/2418929719 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1065+0+0 (secure 0 0 0) 0x7f39f40423e0 con 0x7f3a000f5480 2024-02-20T12:52:34.093 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:34.047+0000 7f39feffd700 1 --2- 172.21.15.71:0/2418929719 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f39e8031090 0x7f39e8033550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:52:34.093 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:34.047+0000 7f39feffd700 1 --2- 172.21.15.71:0/2418929719 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f39e8031090 0x7f39e8033550 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7f39f00096a0 tx=0x7f39f0006b40).ready entity=mgr.14156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:52:34.093 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:34.052+0000 7f39e7fff700 1 -- 172.21.15.71:0/2418929719 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+153237 (secure 0 0 0) 0x7f39f401d7f0 con 0x7f3a000f5480 2024-02-20T12:52:34.260 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:34.259+0000 7f3a05e42700 1 -- 172.21.15.71:0/2418929719 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f3a001081e0 con 0x7f3a000f5480 2024-02-20T12:52:34.261 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:34.260+0000 7f39e7fff700 1 -- 172.21.15.71:0/2418929719 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7f39f401d7f0 con 0x7f3a000f5480 2024-02-20T12:52:34.261 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-20T12:52:34.261 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":1,"fsid":"680ee126-cfee-11ee-95bb-87774f69a715","modified":"2024-02-20T12:49:44.699059Z","created":"2024-02-20T12:49:44.699059Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi064","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:3300","nonce":0},{"type":"v1","addr":"172.21.15.64:6789","nonce":0}]},"addr":"172.21.15.64:6789/0","public_addr":"172.21.15.64:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-20T12:52:34.262 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:34.262+0000 7f3a05e42700 1 -- 172.21.15.71:0/2418929719 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f39e8031090 msgr2=0x7f39e8033550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:52:34.263 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:34.262+0000 7f3a05e42700 1 --2- 172.21.15.71:0/2418929719 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f39e8031090 0x7f39e8033550 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7f39f00096a0 tx=0x7f39f0006b40).stop 2024-02-20T12:52:34.263 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:34.262+0000 7f3a05e42700 1 -- 172.21.15.71:0/2418929719 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3a000f5480 msgr2=0x7f3a001067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:52:34.263 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:34.262+0000 7f3a05e42700 1 --2- 172.21.15.71:0/2418929719 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3a000f5480 0x7f3a001067e0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 rx=0x7f39f40089f0 tx=0x7f39f400b040).stop 2024-02-20T12:52:34.263 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:34.262+0000 7f3a05e42700 1 -- 172.21.15.71:0/2418929719 shutdown_connections 2024-02-20T12:52:34.263 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:34.262+0000 7f3a05e42700 1 --2- 172.21.15.71:0/2418929719 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f39e8031090 0x7f39e8033550 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:52:34.263 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:34.262+0000 7f3a05e42700 1 --2- 172.21.15.71:0/2418929719 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3a000f5480 0x7f3a001067e0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:52:34.263 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:34.262+0000 7f3a05e42700 1 -- 172.21.15.71:0/2418929719 >> 172.21.15.71:0/2418929719 conn(0x7f3a000f0be0 msgr2=0x7f3a0005ff40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:52:34.263 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:34.262+0000 7f3a05e42700 1 -- 172.21.15.71:0/2418929719 shutdown_connections 2024-02-20T12:52:34.263 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:34.262+0000 7f3a05e42700 1 -- 172.21.15.71:0/2418929719 wait complete. 2024-02-20T12:52:34.264 INFO:teuthology.orchestra.run.smithi071.stderr:dumped monmap epoch 1 2024-02-20T12:52:35.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:52:34 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:52:33.717683+0000 mgr.smithi064.vriyor (mgr.14156) 52 : cluster [DBG] pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:52:35.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:52:34 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:52:34.260868+0000 mon.smithi064 (mon.0) 126 : audit [DBG] from='client.? 172.21.15.71:0/2418929719' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-20T12:52:35.985 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-20T12:52:35.985 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/daemon-base:latest-pacific shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 680ee126-cfee-11ee-95bb-87774f69a715 -- ceph mon dump -f json 2024-02-20T12:52:37.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:52:36 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:52:35.717973+0000 mgr.smithi064.vriyor (mgr.14156) 53 : cluster [DBG] pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:52:37.808 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:37.806+0000 7f91dd985700 1 -- 172.21.15.71:0/2185607406 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f91d80f52f0 msgr2=0x7f91d80f5710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:52:37.808 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:37.806+0000 7f91dd985700 1 --2- 172.21.15.71:0/2185607406 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f91d80f52f0 0x7f91d80f5710 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 rx=0x7f91c8009fb0 tx=0x7f91c8009b30).stop 2024-02-20T12:52:37.808 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:37.806+0000 7f91dd985700 1 -- 172.21.15.71:0/2185607406 shutdown_connections 2024-02-20T12:52:37.808 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:37.806+0000 7f91dd985700 1 --2- 172.21.15.71:0/2185607406 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f91d80f52f0 0x7f91d80f5710 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:52:37.808 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:37.806+0000 7f91dd985700 1 -- 172.21.15.71:0/2185607406 >> 172.21.15.71:0/2185607406 conn(0x7f91d80f0a70 msgr2=0x7f91d80f2e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:52:37.809 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:37.806+0000 7f91dd985700 1 -- 172.21.15.71:0/2185607406 shutdown_connections 2024-02-20T12:52:37.809 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:37.806+0000 7f91dd985700 1 -- 172.21.15.71:0/2185607406 wait complete. 2024-02-20T12:52:37.809 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:37.806+0000 7f91dd985700 1 Processor -- start 2024-02-20T12:52:37.809 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:37.806+0000 7f91dd985700 1 -- start start 2024-02-20T12:52:37.809 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:37.806+0000 7f91dd985700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f91d80f52f0 0x7f91d80fd2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:52:37.809 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:37.806+0000 7f91dd985700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91d80fb750 con 0x7f91d80f52f0 2024-02-20T12:52:37.809 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:37.807+0000 7f91d6ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f91d80f52f0 0x7f91d80fd2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:52:37.809 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:37.807+0000 7f91d6ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f91d80f52f0 0x7f91d80fd2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.71:33442/0 (socket says 172.21.15.71:33442) 2024-02-20T12:52:37.809 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:37.807+0000 7f91d6ffd700 1 -- 172.21.15.71:0/1340043299 learned_addr learned my addr 172.21.15.71:0/1340043299 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-02-20T12:52:37.809 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:37.807+0000 7f91d6ffd700 1 -- 172.21.15.71:0/1340043299 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91c8005040 con 0x7f91d80f52f0 2024-02-20T12:52:37.809 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:37.807+0000 7f91d6ffd700 1 --2- 172.21.15.71:0/1340043299 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f91d80f52f0 0x7f91d80fd2d0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 rx=0x7f91c80089f0 tx=0x7f91c800b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:52:37.809 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:37.807+0000 7f91bf7fe700 1 -- 172.21.15.71:0/1340043299 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f91c8009320 con 0x7f91d80f52f0 2024-02-20T12:52:37.809 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:37.807+0000 7f91dd985700 1 -- 172.21.15.71:0/1340043299 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91d80fb8f0 con 0x7f91d80f52f0 2024-02-20T12:52:37.809 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:37.807+0000 7f91bf7fe700 1 -- 172.21.15.71:0/1340043299 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(25 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f91c8013ec0 con 0x7f91d80f52f0 2024-02-20T12:52:37.810 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:37.807+0000 7f91bf7fe700 1 -- 172.21.15.71:0/1340043299 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f91c8014d30 con 0x7f91d80f52f0 2024-02-20T12:52:37.810 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:37.807+0000 7f91dd985700 1 -- 172.21.15.71:0/1340043299 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91d80fbd80 con 0x7f91d80f52f0 2024-02-20T12:52:37.810 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:37.808+0000 7f91bf7fe700 1 -- 172.21.15.71:0/1340043299 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 37151+0+0 (secure 0 0 0) 0x7f91c8014420 con 0x7f91d80f52f0 2024-02-20T12:52:37.811 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:37.808+0000 7f91bf7fe700 1 --2- 172.21.15.71:0/1340043299 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f91c0031090 0x7f91c0033550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:52:37.811 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:37.808+0000 7f91dd985700 1 -- 172.21.15.71:0/1340043299 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91d80591e0 con 0x7f91d80f52f0 2024-02-20T12:52:37.811 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:37.808+0000 7f91bf7fe700 1 -- 172.21.15.71:0/1340043299 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1065+0+0 (secure 0 0 0) 0x7f91c80424a0 con 0x7f91d80f52f0 2024-02-20T12:52:37.811 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:37.808+0000 7f91d67fc700 1 --2- 172.21.15.71:0/1340043299 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f91c0031090 0x7f91c0033550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:52:37.811 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:37.809+0000 7f91d67fc700 1 --2- 172.21.15.71:0/1340043299 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f91c0031090 0x7f91c0033550 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7f91cc0096a0 tx=0x7f91cc006b40).ready entity=mgr.14156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:52:37.813 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:37.813+0000 7f91bf7fe700 1 -- 172.21.15.71:0/1340043299 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+153237 (secure 0 0 0) 0x7f91c8014750 con 0x7f91d80f52f0 2024-02-20T12:52:38.022 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:38.021+0000 7f91dd985700 1 -- 172.21.15.71:0/1340043299 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f91d8045bb0 con 0x7f91d80f52f0 2024-02-20T12:52:38.022 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:38.022+0000 7f91bf7fe700 1 -- 172.21.15.71:0/1340043299 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7f91c801d7f0 con 0x7f91d80f52f0 2024-02-20T12:52:38.023 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-20T12:52:38.023 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":1,"fsid":"680ee126-cfee-11ee-95bb-87774f69a715","modified":"2024-02-20T12:49:44.699059Z","created":"2024-02-20T12:49:44.699059Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi064","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:3300","nonce":0},{"type":"v1","addr":"172.21.15.64:6789","nonce":0}]},"addr":"172.21.15.64:6789/0","public_addr":"172.21.15.64:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-20T12:52:38.024 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:38.023+0000 7f91dd985700 1 -- 172.21.15.71:0/1340043299 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f91c0031090 msgr2=0x7f91c0033550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:52:38.024 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:38.023+0000 7f91dd985700 1 --2- 172.21.15.71:0/1340043299 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f91c0031090 0x7f91c0033550 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7f91cc0096a0 tx=0x7f91cc006b40).stop 2024-02-20T12:52:38.024 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:38.023+0000 7f91dd985700 1 -- 172.21.15.71:0/1340043299 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f91d80f52f0 msgr2=0x7f91d80fd2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:52:38.024 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:38.023+0000 7f91dd985700 1 --2- 172.21.15.71:0/1340043299 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f91d80f52f0 0x7f91d80fd2d0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 rx=0x7f91c80089f0 tx=0x7f91c800b040).stop 2024-02-20T12:52:38.024 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:38.023+0000 7f91dd985700 1 -- 172.21.15.71:0/1340043299 shutdown_connections 2024-02-20T12:52:38.024 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:38.023+0000 7f91dd985700 1 --2- 172.21.15.71:0/1340043299 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f91c0031090 0x7f91c0033550 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:52:38.024 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:38.023+0000 7f91dd985700 1 --2- 172.21.15.71:0/1340043299 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f91d80f52f0 0x7f91d80fd2d0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:52:38.024 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:38.023+0000 7f91dd985700 1 -- 172.21.15.71:0/1340043299 >> 172.21.15.71:0/1340043299 conn(0x7f91d80f0a70 msgr2=0x7f91d805fee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:52:38.024 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:38.023+0000 7f91dd985700 1 -- 172.21.15.71:0/1340043299 shutdown_connections 2024-02-20T12:52:38.024 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:38.023+0000 7f91dd985700 1 -- 172.21.15.71:0/1340043299 wait complete. 2024-02-20T12:52:38.025 INFO:teuthology.orchestra.run.smithi071.stderr:dumped monmap epoch 1 2024-02-20T12:52:39.313 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:52:38 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:52:37.718210+0000 mgr.smithi064.vriyor (mgr.14156) 54 : cluster [DBG] pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:52:39.313 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:52:38 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:52:38.022398+0000 mon.smithi064 (mon.0) 127 : audit [DBG] from='client.? 172.21.15.71:0/1340043299' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-20T12:52:39.715 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-20T12:52:39.715 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/daemon-base:latest-pacific shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 680ee126-cfee-11ee-95bb-87774f69a715 -- ceph mon dump -f json 2024-02-20T12:52:41.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:52:40 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:52:39.718503+0000 mgr.smithi064.vriyor (mgr.14156) 55 : cluster [DBG] pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:52:41.592 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:41.590+0000 7f4be819b700 1 -- 172.21.15.71:0/1079789857 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4be00f5410 msgr2=0x7f4be00f5830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:52:41.592 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:41.590+0000 7f4be819b700 1 --2- 172.21.15.71:0/1079789857 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4be00f5410 0x7f4be00f5830 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 rx=0x7f4bd0009fb0 tx=0x7f4bd0009b30).stop 2024-02-20T12:52:41.592 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:41.590+0000 7f4be819b700 1 -- 172.21.15.71:0/1079789857 shutdown_connections 2024-02-20T12:52:41.592 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:41.590+0000 7f4be819b700 1 --2- 172.21.15.71:0/1079789857 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4be00f5410 0x7f4be00f5830 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:52:41.592 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:41.590+0000 7f4be819b700 1 -- 172.21.15.71:0/1079789857 >> 172.21.15.71:0/1079789857 conn(0x7f4be00f0b70 msgr2=0x7f4be00f2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:52:41.592 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:41.590+0000 7f4be819b700 1 -- 172.21.15.71:0/1079789857 shutdown_connections 2024-02-20T12:52:41.592 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:41.590+0000 7f4be819b700 1 -- 172.21.15.71:0/1079789857 wait complete. 2024-02-20T12:52:41.592 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:41.591+0000 7f4be819b700 1 Processor -- start 2024-02-20T12:52:41.592 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:41.591+0000 7f4be819b700 1 -- start start 2024-02-20T12:52:41.592 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:41.591+0000 7f4be819b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4be00f5410 0x7f4be0108d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:52:41.592 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:41.591+0000 7f4be819b700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4be0109230 con 0x7f4be00f5410 2024-02-20T12:52:41.593 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:41.591+0000 7f4be5f37700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4be00f5410 0x7f4be0108d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:52:41.593 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:41.591+0000 7f4be5f37700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4be00f5410 0x7f4be0108d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.71:33462/0 (socket says 172.21.15.71:33462) 2024-02-20T12:52:41.593 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:41.591+0000 7f4be5f37700 1 -- 172.21.15.71:0/526907960 learned_addr learned my addr 172.21.15.71:0/526907960 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-02-20T12:52:41.593 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:41.591+0000 7f4be5f37700 1 -- 172.21.15.71:0/526907960 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bd0005040 con 0x7f4be00f5410 2024-02-20T12:52:41.594 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:41.592+0000 7f4be5f37700 1 --2- 172.21.15.71:0/526907960 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4be00f5410 0x7f4be0108d40 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 rx=0x7f4bd00089f0 tx=0x7f4bd000b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:52:41.594 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:41.592+0000 7f4bd67fc700 1 -- 172.21.15.71:0/526907960 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4bd0009320 con 0x7f4be00f5410 2024-02-20T12:52:41.594 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:41.592+0000 7f4bd67fc700 1 -- 172.21.15.71:0/526907960 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(25 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f4bd0013ec0 con 0x7f4be00f5410 2024-02-20T12:52:41.594 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:41.592+0000 7f4be819b700 1 -- 172.21.15.71:0/526907960 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4be0109430 con 0x7f4be00f5410 2024-02-20T12:52:41.595 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:41.592+0000 7f4bd67fc700 1 -- 172.21.15.71:0/526907960 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4bd0014d30 con 0x7f4be00f5410 2024-02-20T12:52:41.595 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:41.592+0000 7f4be819b700 1 -- 172.21.15.71:0/526907960 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4be010a040 con 0x7f4be00f5410 2024-02-20T12:52:41.595 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:41.593+0000 7f4bd67fc700 1 -- 172.21.15.71:0/526907960 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 37151+0+0 (secure 0 0 0) 0x7f4bd00095f0 con 0x7f4be00f5410 2024-02-20T12:52:41.595 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:41.593+0000 7f4be819b700 1 -- 172.21.15.71:0/526907960 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4be00591e0 con 0x7f4be00f5410 2024-02-20T12:52:41.595 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:41.593+0000 7f4bd67fc700 1 --2- 172.21.15.71:0/526907960 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f4bcc031090 0x7f4bcc033550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:52:41.595 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:41.593+0000 7f4bd67fc700 1 -- 172.21.15.71:0/526907960 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1065+0+0 (secure 0 0 0) 0x7f4bd00423e0 con 0x7f4be00f5410 2024-02-20T12:52:41.595 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:41.593+0000 7f4be5736700 1 --2- 172.21.15.71:0/526907960 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f4bcc031090 0x7f4bcc033550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:52:41.595 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:41.594+0000 7f4be5736700 1 --2- 172.21.15.71:0/526907960 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f4bcc031090 0x7f4bcc033550 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7f4bdc0096a0 tx=0x7f4bdc006b40).ready entity=mgr.14156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:52:41.598 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:41.598+0000 7f4bd67fc700 1 -- 172.21.15.71:0/526907960 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+153237 (secure 0 0 0) 0x7f4bd00268c0 con 0x7f4be00f5410 2024-02-20T12:52:41.806 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:41.805+0000 7f4be819b700 1 -- 172.21.15.71:0/526907960 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f4be0045bb0 con 0x7f4be00f5410 2024-02-20T12:52:41.806 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:41.806+0000 7f4bd67fc700 1 -- 172.21.15.71:0/526907960 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7f4bd001d7f0 con 0x7f4be00f5410 2024-02-20T12:52:41.806 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-20T12:52:41.807 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":1,"fsid":"680ee126-cfee-11ee-95bb-87774f69a715","modified":"2024-02-20T12:49:44.699059Z","created":"2024-02-20T12:49:44.699059Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi064","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:3300","nonce":0},{"type":"v1","addr":"172.21.15.64:6789","nonce":0}]},"addr":"172.21.15.64:6789/0","public_addr":"172.21.15.64:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-20T12:52:41.808 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:41.807+0000 7f4be819b700 1 -- 172.21.15.71:0/526907960 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f4bcc031090 msgr2=0x7f4bcc033550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:52:41.808 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:41.807+0000 7f4be819b700 1 --2- 172.21.15.71:0/526907960 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f4bcc031090 0x7f4bcc033550 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7f4bdc0096a0 tx=0x7f4bdc006b40).stop 2024-02-20T12:52:41.809 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:41.807+0000 7f4be819b700 1 -- 172.21.15.71:0/526907960 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4be00f5410 msgr2=0x7f4be0108d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:52:41.809 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:41.808+0000 7f4be819b700 1 --2- 172.21.15.71:0/526907960 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4be00f5410 0x7f4be0108d40 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 rx=0x7f4bd00089f0 tx=0x7f4bd000b040).stop 2024-02-20T12:52:41.809 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:41.808+0000 7f4be819b700 1 -- 172.21.15.71:0/526907960 shutdown_connections 2024-02-20T12:52:41.809 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:41.808+0000 7f4be819b700 1 --2- 172.21.15.71:0/526907960 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f4bcc031090 0x7f4bcc033550 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:52:41.809 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:41.808+0000 7f4be819b700 1 --2- 172.21.15.71:0/526907960 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4be00f5410 0x7f4be0108d40 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:52:41.809 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:41.808+0000 7f4be819b700 1 -- 172.21.15.71:0/526907960 >> 172.21.15.71:0/526907960 conn(0x7f4be00f0b70 msgr2=0x7f4be005ff80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:52:41.809 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:41.808+0000 7f4be819b700 1 -- 172.21.15.71:0/526907960 shutdown_connections 2024-02-20T12:52:41.809 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:41.808+0000 7f4be819b700 1 -- 172.21.15.71:0/526907960 wait complete. 2024-02-20T12:52:41.809 INFO:teuthology.orchestra.run.smithi071.stderr:dumped monmap epoch 1 2024-02-20T12:52:42.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:52:41 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:52:41.806814+0000 mon.smithi064 (mon.0) 128 : audit [DBG] from='client.? 172.21.15.71:0/526907960' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-20T12:52:43.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:52:42 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:52:41.718836+0000 mgr.smithi064.vriyor (mgr.14156) 56 : cluster [DBG] pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:52:43.377 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-20T12:52:43.378 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/daemon-base:latest-pacific shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 680ee126-cfee-11ee-95bb-87774f69a715 -- ceph mon dump -f json 2024-02-20T12:52:45.250 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:45.248+0000 7f7c8a8f6700 1 -- 172.21.15.71:0/1347524460 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7c840f6590 msgr2=0x7f7c840f69b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:52:45.251 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:45.248+0000 7f7c8a8f6700 1 --2- 172.21.15.71:0/1347524460 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7c840f6590 0x7f7c840f69b0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 rx=0x7f7c74004760 tx=0x7f7c74009ad0).stop 2024-02-20T12:52:45.251 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:45.248+0000 7f7c8a8f6700 1 -- 172.21.15.71:0/1347524460 shutdown_connections 2024-02-20T12:52:45.251 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:45.248+0000 7f7c8a8f6700 1 --2- 172.21.15.71:0/1347524460 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7c840f6590 0x7f7c840f69b0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:52:45.251 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:45.248+0000 7f7c8a8f6700 1 -- 172.21.15.71:0/1347524460 >> 172.21.15.71:0/1347524460 conn(0x7f7c840f1cd0 msgr2=0x7f7c840f4110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:52:45.251 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:45.248+0000 7f7c8a8f6700 1 -- 172.21.15.71:0/1347524460 shutdown_connections 2024-02-20T12:52:45.251 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:45.248+0000 7f7c8a8f6700 1 -- 172.21.15.71:0/1347524460 wait complete. 2024-02-20T12:52:45.251 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:45.249+0000 7f7c8a8f6700 1 Processor -- start 2024-02-20T12:52:45.251 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:45.249+0000 7f7c8a8f6700 1 -- start start 2024-02-20T12:52:45.251 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:45.249+0000 7f7c8a8f6700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7c840f6590 0x7f7c841067c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:52:45.252 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:45.249+0000 7f7c8a8f6700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c84106cb0 con 0x7f7c840f6590 2024-02-20T12:52:45.252 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:45.249+0000 7f7c83fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7c840f6590 0x7f7c841067c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:52:45.252 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:45.249+0000 7f7c83fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7c840f6590 0x7f7c841067c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.71:33480/0 (socket says 172.21.15.71:33480) 2024-02-20T12:52:45.252 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:45.249+0000 7f7c83fff700 1 -- 172.21.15.71:0/3215152814 learned_addr learned my addr 172.21.15.71:0/3215152814 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-02-20T12:52:45.252 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:45.249+0000 7f7c83fff700 1 -- 172.21.15.71:0/3215152814 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c74009d70 con 0x7f7c840f6590 2024-02-20T12:52:45.252 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:45.250+0000 7f7c83fff700 1 --2- 172.21.15.71:0/3215152814 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7c840f6590 0x7f7c841067c0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 rx=0x7f7c74004170 tx=0x7f7c74009ad0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:52:45.252 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:45.250+0000 7f7c80ff9700 1 -- 172.21.15.71:0/3215152814 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7c7400e040 con 0x7f7c840f6590 2024-02-20T12:52:45.252 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:45.250+0000 7f7c8a8f6700 1 -- 172.21.15.71:0/3215152814 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c84106eb0 con 0x7f7c840f6590 2024-02-20T12:52:45.252 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:45.250+0000 7f7c80ff9700 1 -- 172.21.15.71:0/3215152814 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(25 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f7c74008ef0 con 0x7f7c840f6590 2024-02-20T12:52:45.252 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:45.250+0000 7f7c80ff9700 1 -- 172.21.15.71:0/3215152814 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7c74014c90 con 0x7f7c840f6590 2024-02-20T12:52:45.252 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:45.250+0000 7f7c8a8f6700 1 -- 172.21.15.71:0/3215152814 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c84107340 con 0x7f7c840f6590 2024-02-20T12:52:45.253 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:45.251+0000 7f7c80ff9700 1 -- 172.21.15.71:0/3215152814 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 37151+0+0 (secure 0 0 0) 0x7f7c74014420 con 0x7f7c840f6590 2024-02-20T12:52:45.253 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:45.251+0000 7f7c8a8f6700 1 -- 172.21.15.71:0/3215152814 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c84045bb0 con 0x7f7c840f6590 2024-02-20T12:52:45.253 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:45.251+0000 7f7c80ff9700 1 --2- 172.21.15.71:0/3215152814 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f7c6c031090 0x7f7c6c033550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:52:45.254 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:45.251+0000 7f7c80ff9700 1 -- 172.21.15.71:0/3215152814 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1065+0+0 (secure 0 0 0) 0x7f7c740422e0 con 0x7f7c840f6590 2024-02-20T12:52:45.254 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:45.251+0000 7f7c837fe700 1 --2- 172.21.15.71:0/3215152814 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f7c6c031090 0x7f7c6c033550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:52:45.254 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:45.251+0000 7f7c837fe700 1 --2- 172.21.15.71:0/3215152814 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f7c6c031090 0x7f7c6c033550 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7f7c780096a0 tx=0x7f7c78006b40).ready entity=mgr.14156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:52:45.256 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:45.256+0000 7f7c80ff9700 1 -- 172.21.15.71:0/3215152814 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+153237 (secure 0 0 0) 0x7f7c74012d00 con 0x7f7c840f6590 2024-02-20T12:52:45.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:52:44 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:52:43.719181+0000 mgr.smithi064.vriyor (mgr.14156) 57 : cluster [DBG] pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:52:45.462 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:45.461+0000 7f7c8a8f6700 1 -- 172.21.15.71:0/3215152814 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f7c84108170 con 0x7f7c840f6590 2024-02-20T12:52:45.462 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:45.462+0000 7f7c80ff9700 1 -- 172.21.15.71:0/3215152814 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7f7c74012b20 con 0x7f7c840f6590 2024-02-20T12:52:45.463 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-20T12:52:45.463 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":1,"fsid":"680ee126-cfee-11ee-95bb-87774f69a715","modified":"2024-02-20T12:49:44.699059Z","created":"2024-02-20T12:49:44.699059Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi064","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:3300","nonce":0},{"type":"v1","addr":"172.21.15.64:6789","nonce":0}]},"addr":"172.21.15.64:6789/0","public_addr":"172.21.15.64:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-20T12:52:45.464 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:45.464+0000 7f7c8a8f6700 1 -- 172.21.15.71:0/3215152814 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f7c6c031090 msgr2=0x7f7c6c033550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:52:45.464 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:45.464+0000 7f7c8a8f6700 1 --2- 172.21.15.71:0/3215152814 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f7c6c031090 0x7f7c6c033550 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7f7c780096a0 tx=0x7f7c78006b40).stop 2024-02-20T12:52:45.464 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:45.464+0000 7f7c8a8f6700 1 -- 172.21.15.71:0/3215152814 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7c840f6590 msgr2=0x7f7c841067c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:52:45.464 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:45.464+0000 7f7c8a8f6700 1 --2- 172.21.15.71:0/3215152814 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7c840f6590 0x7f7c841067c0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 rx=0x7f7c74004170 tx=0x7f7c74009ad0).stop 2024-02-20T12:52:45.464 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:45.464+0000 7f7c8a8f6700 1 -- 172.21.15.71:0/3215152814 shutdown_connections 2024-02-20T12:52:45.465 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:45.464+0000 7f7c8a8f6700 1 --2- 172.21.15.71:0/3215152814 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f7c6c031090 0x7f7c6c033550 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:52:45.465 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:45.464+0000 7f7c8a8f6700 1 --2- 172.21.15.71:0/3215152814 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7c840f6590 0x7f7c841067c0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:52:45.465 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:45.464+0000 7f7c8a8f6700 1 -- 172.21.15.71:0/3215152814 >> 172.21.15.71:0/3215152814 conn(0x7f7c840f1cd0 msgr2=0x7f7c840f27c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:52:45.465 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:45.464+0000 7f7c8a8f6700 1 -- 172.21.15.71:0/3215152814 shutdown_connections 2024-02-20T12:52:45.465 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:45.464+0000 7f7c8a8f6700 1 -- 172.21.15.71:0/3215152814 wait complete. 2024-02-20T12:52:45.465 INFO:teuthology.orchestra.run.smithi071.stderr:dumped monmap epoch 1 2024-02-20T12:52:46.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:52:45 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:52:45.463023+0000 mon.smithi064 (mon.0) 129 : audit [DBG] from='client.? 172.21.15.71:0/3215152814' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-20T12:52:47.008 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-20T12:52:47.008 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/daemon-base:latest-pacific shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 680ee126-cfee-11ee-95bb-87774f69a715 -- ceph mon dump -f json 2024-02-20T12:52:47.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:52:46 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:52:45.719557+0000 mgr.smithi064.vriyor (mgr.14156) 58 : cluster [DBG] pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:52:49.089 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:49.087+0000 7fab46081700 1 -- 172.21.15.71:0/2705386686 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fab400f5480 msgr2=0x7fab400f58a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:52:49.090 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:49.087+0000 7fab46081700 1 --2- 172.21.15.71:0/2705386686 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fab400f5480 0x7fab400f58a0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 rx=0x7fab34009fb0 tx=0x7fab34009b30).stop 2024-02-20T12:52:49.090 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:49.087+0000 7fab46081700 1 -- 172.21.15.71:0/2705386686 shutdown_connections 2024-02-20T12:52:49.090 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:49.087+0000 7fab46081700 1 --2- 172.21.15.71:0/2705386686 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fab400f5480 0x7fab400f58a0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:52:49.090 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:49.087+0000 7fab46081700 1 -- 172.21.15.71:0/2705386686 >> 172.21.15.71:0/2705386686 conn(0x7fab400f0be0 msgr2=0x7fab400f3000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:52:49.090 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:49.087+0000 7fab46081700 1 -- 172.21.15.71:0/2705386686 shutdown_connections 2024-02-20T12:52:49.090 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:49.087+0000 7fab46081700 1 -- 172.21.15.71:0/2705386686 wait complete. 2024-02-20T12:52:49.090 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:49.088+0000 7fab46081700 1 Processor -- start 2024-02-20T12:52:49.090 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:49.088+0000 7fab46081700 1 -- start start 2024-02-20T12:52:49.090 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:49.088+0000 7fab46081700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fab400f5480 0x7fab401094f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:52:49.090 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:49.088+0000 7fab46081700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab40060320 con 0x7fab400f5480 2024-02-20T12:52:49.091 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:49.088+0000 7fab3f7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fab400f5480 0x7fab401094f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:52:49.091 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:49.088+0000 7fab3f7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fab400f5480 0x7fab401094f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.71:48514/0 (socket says 172.21.15.71:48514) 2024-02-20T12:52:49.091 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:49.088+0000 7fab3f7fe700 1 -- 172.21.15.71:0/2631794134 learned_addr learned my addr 172.21.15.71:0/2631794134 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-02-20T12:52:49.091 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:49.088+0000 7fab3f7fe700 1 -- 172.21.15.71:0/2631794134 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab34005040 con 0x7fab400f5480 2024-02-20T12:52:49.091 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:49.089+0000 7fab3f7fe700 1 --2- 172.21.15.71:0/2631794134 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fab400f5480 0x7fab401094f0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 rx=0x7fab340089f0 tx=0x7fab3400b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:52:49.091 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:49.089+0000 7fab27fff700 1 -- 172.21.15.71:0/2631794134 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fab34009320 con 0x7fab400f5480 2024-02-20T12:52:49.091 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:49.089+0000 7fab46081700 1 -- 172.21.15.71:0/2631794134 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab40060520 con 0x7fab400f5480 2024-02-20T12:52:49.091 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:49.089+0000 7fab27fff700 1 -- 172.21.15.71:0/2631794134 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(25 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7fab34013ec0 con 0x7fab400f5480 2024-02-20T12:52:49.091 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:49.089+0000 7fab27fff700 1 -- 172.21.15.71:0/2631794134 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fab34014d30 con 0x7fab400f5480 2024-02-20T12:52:49.091 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:49.089+0000 7fab46081700 1 -- 172.21.15.71:0/2631794134 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab40109ba0 con 0x7fab400f5480 2024-02-20T12:52:49.092 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:49.090+0000 7fab27fff700 1 -- 172.21.15.71:0/2631794134 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 37151+0+0 (secure 0 0 0) 0x7fab340263a0 con 0x7fab400f5480 2024-02-20T12:52:49.093 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:49.090+0000 7fab27fff700 1 --2- 172.21.15.71:0/2631794134 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fab28031090 0x7fab28033550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:52:49.093 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:49.090+0000 7fab27fff700 1 -- 172.21.15.71:0/2631794134 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1065+0+0 (secure 0 0 0) 0x7fab34042450 con 0x7fab400f5480 2024-02-20T12:52:49.093 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:49.090+0000 7fab46081700 1 -- 172.21.15.71:0/2631794134 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab400591e0 con 0x7fab400f5480 2024-02-20T12:52:49.093 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:49.090+0000 7fab3effd700 1 --2- 172.21.15.71:0/2631794134 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fab28031090 0x7fab28033550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:52:49.093 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:49.090+0000 7fab3effd700 1 --2- 172.21.15.71:0/2631794134 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fab28031090 0x7fab28033550 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7fab300096a0 tx=0x7fab30006b40).ready entity=mgr.14156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:52:49.095 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:49.095+0000 7fab27fff700 1 -- 172.21.15.71:0/2631794134 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+153237 (secure 0 0 0) 0x7fab34012920 con 0x7fab400f5480 2024-02-20T12:52:49.307 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:49.306+0000 7fab46081700 1 -- 172.21.15.71:0/2631794134 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fab400faf70 con 0x7fab400f5480 2024-02-20T12:52:49.308 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:49.307+0000 7fab27fff700 1 -- 172.21.15.71:0/2631794134 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7fab3402b020 con 0x7fab400f5480 2024-02-20T12:52:49.308 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-20T12:52:49.308 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":1,"fsid":"680ee126-cfee-11ee-95bb-87774f69a715","modified":"2024-02-20T12:49:44.699059Z","created":"2024-02-20T12:49:44.699059Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi064","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:3300","nonce":0},{"type":"v1","addr":"172.21.15.64:6789","nonce":0}]},"addr":"172.21.15.64:6789/0","public_addr":"172.21.15.64:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-20T12:52:49.309 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:49.309+0000 7fab46081700 1 -- 172.21.15.71:0/2631794134 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fab28031090 msgr2=0x7fab28033550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:52:49.309 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:49.309+0000 7fab46081700 1 --2- 172.21.15.71:0/2631794134 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fab28031090 0x7fab28033550 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7fab300096a0 tx=0x7fab30006b40).stop 2024-02-20T12:52:49.309 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:49.309+0000 7fab46081700 1 -- 172.21.15.71:0/2631794134 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fab400f5480 msgr2=0x7fab401094f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:52:49.309 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:49.309+0000 7fab46081700 1 --2- 172.21.15.71:0/2631794134 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fab400f5480 0x7fab401094f0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 rx=0x7fab340089f0 tx=0x7fab3400b040).stop 2024-02-20T12:52:49.310 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:49.309+0000 7fab46081700 1 -- 172.21.15.71:0/2631794134 shutdown_connections 2024-02-20T12:52:49.310 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:49.309+0000 7fab46081700 1 --2- 172.21.15.71:0/2631794134 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fab28031090 0x7fab28033550 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:52:49.310 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:49.309+0000 7fab46081700 1 --2- 172.21.15.71:0/2631794134 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fab400f5480 0x7fab401094f0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:52:49.310 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:49.309+0000 7fab46081700 1 -- 172.21.15.71:0/2631794134 >> 172.21.15.71:0/2631794134 conn(0x7fab400f0be0 msgr2=0x7fab400f16d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:52:49.310 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:49.309+0000 7fab46081700 1 -- 172.21.15.71:0/2631794134 shutdown_connections 2024-02-20T12:52:49.310 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:49.309+0000 7fab46081700 1 -- 172.21.15.71:0/2631794134 wait complete. 2024-02-20T12:52:49.310 INFO:teuthology.orchestra.run.smithi071.stderr:dumped monmap epoch 1 2024-02-20T12:52:49.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:52:48 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:52:47.719920+0000 mgr.smithi064.vriyor (mgr.14156) 59 : cluster [DBG] pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:52:50.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:52:49 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:52:49.307928+0000 mon.smithi064 (mon.0) 130 : audit [DBG] from='client.? 172.21.15.71:0/2631794134' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-20T12:52:50.911 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-20T12:52:50.911 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/daemon-base:latest-pacific shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 680ee126-cfee-11ee-95bb-87774f69a715 -- ceph mon dump -f json 2024-02-20T12:52:51.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:52:50 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:52:49.720226+0000 mgr.smithi064.vriyor (mgr.14156) 60 : cluster [DBG] pgmap v52: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:52:52.595 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:52.592+0000 7f8ab48dc700 1 -- 172.21.15.71:0/1310946680 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8ab00f5480 msgr2=0x7f8ab00f58a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:52:52.595 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:52.592+0000 7f8ab48dc700 1 --2- 172.21.15.71:0/1310946680 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8ab00f5480 0x7f8ab00f58a0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 rx=0x7f8aa0009fb0 tx=0x7f8aa0009b30).stop 2024-02-20T12:52:52.596 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:52.592+0000 7f8ab48dc700 1 -- 172.21.15.71:0/1310946680 shutdown_connections 2024-02-20T12:52:52.596 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:52.592+0000 7f8ab48dc700 1 --2- 172.21.15.71:0/1310946680 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8ab00f5480 0x7f8ab00f58a0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:52:52.596 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:52.592+0000 7f8ab48dc700 1 -- 172.21.15.71:0/1310946680 >> 172.21.15.71:0/1310946680 conn(0x7f8ab00f0be0 msgr2=0x7f8ab00f3000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:52:52.596 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:52.592+0000 7f8ab48dc700 1 -- 172.21.15.71:0/1310946680 shutdown_connections 2024-02-20T12:52:52.596 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:52.592+0000 7f8ab48dc700 1 -- 172.21.15.71:0/1310946680 wait complete. 2024-02-20T12:52:52.596 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:52.592+0000 7f8ab48dc700 1 Processor -- start 2024-02-20T12:52:52.596 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:52.592+0000 7f8ab48dc700 1 -- start start 2024-02-20T12:52:52.596 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:52.592+0000 7f8ab48dc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8ab00f5480 0x7f8ab00fd460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:52:52.596 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:52.592+0000 7f8ab48dc700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ab00fb8e0 con 0x7f8ab00f5480 2024-02-20T12:52:52.596 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:52.593+0000 7f8aae59c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8ab00f5480 0x7f8ab00fd460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:52:52.596 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:52.593+0000 7f8aae59c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8ab00f5480 0x7f8ab00fd460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.71:48530/0 (socket says 172.21.15.71:48530) 2024-02-20T12:52:52.596 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:52.593+0000 7f8aae59c700 1 -- 172.21.15.71:0/792173153 learned_addr learned my addr 172.21.15.71:0/792173153 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-02-20T12:52:52.597 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:52.593+0000 7f8aae59c700 1 -- 172.21.15.71:0/792173153 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8aa0005040 con 0x7f8ab00f5480 2024-02-20T12:52:52.597 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:52.593+0000 7f8aae59c700 1 --2- 172.21.15.71:0/792173153 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8ab00f5480 0x7f8ab00fd460 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 rx=0x7f8aa00089f0 tx=0x7f8aa000b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:52:52.597 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:52.593+0000 7f8a9effd700 1 -- 172.21.15.71:0/792173153 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8aa0009320 con 0x7f8ab00f5480 2024-02-20T12:52:52.597 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:52.593+0000 7f8ab48dc700 1 -- 172.21.15.71:0/792173153 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ab00fba80 con 0x7f8ab00f5480 2024-02-20T12:52:52.597 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:52.593+0000 7f8a9effd700 1 -- 172.21.15.71:0/792173153 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(25 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f8aa0013ec0 con 0x7f8ab00f5480 2024-02-20T12:52:52.597 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:52.593+0000 7f8a9effd700 1 -- 172.21.15.71:0/792173153 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8aa0014d30 con 0x7f8ab00f5480 2024-02-20T12:52:52.597 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:52.593+0000 7f8ab48dc700 1 -- 172.21.15.71:0/792173153 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ab00fbf10 con 0x7f8ab00f5480 2024-02-20T12:52:52.597 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:52.594+0000 7f8a9effd700 1 -- 172.21.15.71:0/792173153 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 37151+0+0 (secure 0 0 0) 0x7f8aa00095f0 con 0x7f8ab00f5480 2024-02-20T12:52:52.598 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:52.594+0000 7f8a9effd700 1 --2- 172.21.15.71:0/792173153 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f8a98031090 0x7f8a98033550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:52:52.598 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:52.594+0000 7f8ab48dc700 1 -- 172.21.15.71:0/792173153 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ab00591e0 con 0x7f8ab00f5480 2024-02-20T12:52:52.598 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:52.594+0000 7f8a9effd700 1 -- 172.21.15.71:0/792173153 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1065+0+0 (secure 0 0 0) 0x7f8aa00423e0 con 0x7f8ab00f5480 2024-02-20T12:52:52.598 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:52.594+0000 7f8aadd9b700 1 --2- 172.21.15.71:0/792173153 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f8a98031090 0x7f8a98033550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:52:52.598 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:52.595+0000 7f8aadd9b700 1 --2- 172.21.15.71:0/792173153 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f8a98031090 0x7f8a98033550 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7f8aa40096a0 tx=0x7f8aa4006b40).ready entity=mgr.14156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:52:52.601 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:52.599+0000 7f8a9effd700 1 -- 172.21.15.71:0/792173153 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+153237 (secure 0 0 0) 0x7f8aa001d7f0 con 0x7f8ab00f5480 2024-02-20T12:52:52.809 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:52.807+0000 7f8ab48dc700 1 -- 172.21.15.71:0/792173153 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f8ab00faf70 con 0x7f8ab00f5480 2024-02-20T12:52:52.809 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:52.808+0000 7f8a9effd700 1 -- 172.21.15.71:0/792173153 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7f8aa0007da0 con 0x7f8ab00f5480 2024-02-20T12:52:52.809 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-20T12:52:52.809 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":1,"fsid":"680ee126-cfee-11ee-95bb-87774f69a715","modified":"2024-02-20T12:49:44.699059Z","created":"2024-02-20T12:49:44.699059Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi064","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:3300","nonce":0},{"type":"v1","addr":"172.21.15.64:6789","nonce":0}]},"addr":"172.21.15.64:6789/0","public_addr":"172.21.15.64:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-20T12:52:52.811 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:52.809+0000 7f8ab48dc700 1 -- 172.21.15.71:0/792173153 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f8a98031090 msgr2=0x7f8a98033550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:52:52.811 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:52.809+0000 7f8ab48dc700 1 --2- 172.21.15.71:0/792173153 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f8a98031090 0x7f8a98033550 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7f8aa40096a0 tx=0x7f8aa4006b40).stop 2024-02-20T12:52:52.811 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:52.809+0000 7f8ab48dc700 1 -- 172.21.15.71:0/792173153 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8ab00f5480 msgr2=0x7f8ab00fd460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:52:52.811 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:52.809+0000 7f8ab48dc700 1 --2- 172.21.15.71:0/792173153 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8ab00f5480 0x7f8ab00fd460 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 rx=0x7f8aa00089f0 tx=0x7f8aa000b040).stop 2024-02-20T12:52:52.811 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:52.809+0000 7f8ab48dc700 1 -- 172.21.15.71:0/792173153 shutdown_connections 2024-02-20T12:52:52.812 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:52.809+0000 7f8ab48dc700 1 --2- 172.21.15.71:0/792173153 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f8a98031090 0x7f8a98033550 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:52:52.812 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:52.809+0000 7f8ab48dc700 1 --2- 172.21.15.71:0/792173153 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8ab00f5480 0x7f8ab00fd460 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:52:52.812 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:52.809+0000 7f8ab48dc700 1 -- 172.21.15.71:0/792173153 >> 172.21.15.71:0/792173153 conn(0x7f8ab00f0be0 msgr2=0x7f8ab005ff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:52:52.812 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:52.809+0000 7f8ab48dc700 1 -- 172.21.15.71:0/792173153 shutdown_connections 2024-02-20T12:52:52.812 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:52.810+0000 7f8ab48dc700 1 -- 172.21.15.71:0/792173153 wait complete. 2024-02-20T12:52:52.812 INFO:teuthology.orchestra.run.smithi071.stderr:dumped monmap epoch 1 2024-02-20T12:52:53.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:52:52 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:52:51.720512+0000 mgr.smithi064.vriyor (mgr.14156) 61 : cluster [DBG] pgmap v53: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:52:53.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:52:52 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:52:52.809490+0000 mon.smithi064 (mon.0) 131 : audit [DBG] from='client.? 172.21.15.71:0/792173153' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-20T12:52:54.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:52:54 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:52:53.844241+0000 mon.smithi064 (mon.0) 132 : audit [INF] from='mgr.14156 172.21.15.64:0/3284974785' entity='mgr.smithi064.vriyor' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi064.vriyor/mirror_snapshot_schedule"}]: dispatch 2024-02-20T12:52:54.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:52:54 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:52:53.845120+0000 mon.smithi064 (mon.0) 133 : audit [INF] from='mgr.14156 172.21.15.64:0/3284974785' entity='mgr.smithi064.vriyor' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi064.vriyor/trash_purge_schedule"}]: dispatch 2024-02-20T12:52:54.396 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-20T12:52:54.397 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/daemon-base:latest-pacific shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 680ee126-cfee-11ee-95bb-87774f69a715 -- ceph mon dump -f json 2024-02-20T12:52:55.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:52:55 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:52:53.720744+0000 mgr.smithi064.vriyor (mgr.14156) 62 : cluster [DBG] pgmap v54: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:52:56.300 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:56.262+0000 7ff13e76b700 1 -- 172.21.15.71:0/711135743 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff1380f5480 msgr2=0x7ff1380f58a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:52:56.300 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:56.262+0000 7ff13e76b700 1 --2- 172.21.15.71:0/711135743 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff1380f5480 0x7ff1380f58a0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 rx=0x7ff128009fb0 tx=0x7ff128009b30).stop 2024-02-20T12:52:56.300 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:56.262+0000 7ff13e76b700 1 -- 172.21.15.71:0/711135743 shutdown_connections 2024-02-20T12:52:56.301 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:56.262+0000 7ff13e76b700 1 --2- 172.21.15.71:0/711135743 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff1380f5480 0x7ff1380f58a0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:52:56.301 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:56.262+0000 7ff13e76b700 1 -- 172.21.15.71:0/711135743 >> 172.21.15.71:0/711135743 conn(0x7ff1380f0be0 msgr2=0x7ff1380f3000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:52:56.301 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:56.262+0000 7ff13e76b700 1 -- 172.21.15.71:0/711135743 shutdown_connections 2024-02-20T12:52:56.301 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:56.262+0000 7ff13e76b700 1 -- 172.21.15.71:0/711135743 wait complete. 2024-02-20T12:52:56.301 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:56.262+0000 7ff13e76b700 1 Processor -- start 2024-02-20T12:52:56.301 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:56.262+0000 7ff13e76b700 1 -- start start 2024-02-20T12:52:56.301 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:56.263+0000 7ff13e76b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff1380f5480 0x7ff1380fd460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:52:56.301 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:56.263+0000 7ff13e76b700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1380fb8e0 con 0x7ff1380f5480 2024-02-20T12:52:56.301 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:56.263+0000 7ff137fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff1380f5480 0x7ff1380fd460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:52:56.301 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:56.263+0000 7ff137fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff1380f5480 0x7ff1380fd460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.71:48536/0 (socket says 172.21.15.71:48536) 2024-02-20T12:52:56.301 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:56.263+0000 7ff137fff700 1 -- 172.21.15.71:0/3083556177 learned_addr learned my addr 172.21.15.71:0/3083556177 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-02-20T12:52:56.302 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:56.263+0000 7ff137fff700 1 -- 172.21.15.71:0/3083556177 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff128005040 con 0x7ff1380f5480 2024-02-20T12:52:56.302 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:56.263+0000 7ff137fff700 1 --2- 172.21.15.71:0/3083556177 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff1380f5480 0x7ff1380fd460 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 rx=0x7ff1280089f0 tx=0x7ff12800b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:52:56.302 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:56.263+0000 7ff134ff9700 1 -- 172.21.15.71:0/3083556177 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff128009320 con 0x7ff1380f5480 2024-02-20T12:52:56.302 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:56.263+0000 7ff13e76b700 1 -- 172.21.15.71:0/3083556177 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1380fba80 con 0x7ff1380f5480 2024-02-20T12:52:56.302 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:56.263+0000 7ff134ff9700 1 -- 172.21.15.71:0/3083556177 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(25 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7ff128013ec0 con 0x7ff1380f5480 2024-02-20T12:52:56.302 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:56.263+0000 7ff134ff9700 1 -- 172.21.15.71:0/3083556177 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff128014d30 con 0x7ff1380f5480 2024-02-20T12:52:56.302 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:56.263+0000 7ff13e76b700 1 -- 172.21.15.71:0/3083556177 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1380fbf10 con 0x7ff1380f5480 2024-02-20T12:52:56.302 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:56.264+0000 7ff134ff9700 1 -- 172.21.15.71:0/3083556177 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 37151+0+0 (secure 0 0 0) 0x7ff128014420 con 0x7ff1380f5480 2024-02-20T12:52:56.302 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:56.264+0000 7ff134ff9700 1 --2- 172.21.15.71:0/3083556177 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7ff120031090 0x7ff120033550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:52:56.302 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:56.264+0000 7ff134ff9700 1 -- 172.21.15.71:0/3083556177 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1065+0+0 (secure 0 0 0) 0x7ff1280423e0 con 0x7ff1380f5480 2024-02-20T12:52:56.302 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:56.264+0000 7ff13e76b700 1 -- 172.21.15.71:0/3083556177 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1380591e0 con 0x7ff1380f5480 2024-02-20T12:52:56.302 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:56.265+0000 7ff1377fe700 1 --2- 172.21.15.71:0/3083556177 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7ff120031090 0x7ff120033550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:52:56.302 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:56.265+0000 7ff1377fe700 1 --2- 172.21.15.71:0/3083556177 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7ff120031090 0x7ff120033550 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7ff12c0096a0 tx=0x7ff12c006b40).ready entity=mgr.14156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:52:56.303 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:56.269+0000 7ff134ff9700 1 -- 172.21.15.71:0/3083556177 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+153237 (secure 0 0 0) 0x7ff128014750 con 0x7ff1380f5480 2024-02-20T12:52:56.475 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:56.474+0000 7ff13e76b700 1 -- 172.21.15.71:0/3083556177 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7ff138045bb0 con 0x7ff1380f5480 2024-02-20T12:52:56.476 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:56.475+0000 7ff134ff9700 1 -- 172.21.15.71:0/3083556177 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7ff12801d7f0 con 0x7ff1380f5480 2024-02-20T12:52:56.476 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-20T12:52:56.476 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":1,"fsid":"680ee126-cfee-11ee-95bb-87774f69a715","modified":"2024-02-20T12:49:44.699059Z","created":"2024-02-20T12:49:44.699059Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi064","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:3300","nonce":0},{"type":"v1","addr":"172.21.15.64:6789","nonce":0}]},"addr":"172.21.15.64:6789/0","public_addr":"172.21.15.64:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-20T12:52:56.478 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:56.476+0000 7ff13e76b700 1 -- 172.21.15.71:0/3083556177 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7ff120031090 msgr2=0x7ff120033550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:52:56.479 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:56.476+0000 7ff13e76b700 1 --2- 172.21.15.71:0/3083556177 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7ff120031090 0x7ff120033550 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7ff12c0096a0 tx=0x7ff12c006b40).stop 2024-02-20T12:52:56.479 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:56.476+0000 7ff13e76b700 1 -- 172.21.15.71:0/3083556177 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff1380f5480 msgr2=0x7ff1380fd460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:52:56.479 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:56.476+0000 7ff13e76b700 1 --2- 172.21.15.71:0/3083556177 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff1380f5480 0x7ff1380fd460 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 rx=0x7ff1280089f0 tx=0x7ff12800b040).stop 2024-02-20T12:52:56.479 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:56.476+0000 7ff13e76b700 1 -- 172.21.15.71:0/3083556177 shutdown_connections 2024-02-20T12:52:56.479 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:56.476+0000 7ff13e76b700 1 --2- 172.21.15.71:0/3083556177 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7ff120031090 0x7ff120033550 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:52:56.479 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:56.476+0000 7ff13e76b700 1 --2- 172.21.15.71:0/3083556177 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff1380f5480 0x7ff1380fd460 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:52:56.480 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:56.476+0000 7ff13e76b700 1 -- 172.21.15.71:0/3083556177 >> 172.21.15.71:0/3083556177 conn(0x7ff1380f0be0 msgr2=0x7ff13805ff40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:52:56.480 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:56.476+0000 7ff13e76b700 1 -- 172.21.15.71:0/3083556177 shutdown_connections 2024-02-20T12:52:56.480 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:56.477+0000 7ff13e76b700 1 -- 172.21.15.71:0/3083556177 wait complete. 2024-02-20T12:52:56.480 INFO:teuthology.orchestra.run.smithi071.stderr:dumped monmap epoch 1 2024-02-20T12:52:57.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:52:57 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:52:55.721124+0000 mgr.smithi064.vriyor (mgr.14156) 63 : cluster [DBG] pgmap v55: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:52:57.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:52:57 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:52:56.476317+0000 mon.smithi064 (mon.0) 134 : audit [DBG] from='client.? 172.21.15.71:0/3083556177' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-20T12:52:58.035 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-20T12:52:58.035 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/daemon-base:latest-pacific shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 680ee126-cfee-11ee-95bb-87774f69a715 -- ceph mon dump -f json 2024-02-20T12:52:59.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:52:59 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:52:57.721495+0000 mgr.smithi064.vriyor (mgr.14156) 64 : cluster [DBG] pgmap v56: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:52:59.923 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:59.870+0000 7f3bff1fd700 1 -- 172.21.15.71:0/1628873906 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3bf80f54b0 msgr2=0x7f3bf80f58d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:52:59.923 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:59.870+0000 7f3bff1fd700 1 --2- 172.21.15.71:0/1628873906 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3bf80f54b0 0x7f3bf80f58d0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 rx=0x7f3bec009fb0 tx=0x7f3bec009b30).stop 2024-02-20T12:52:59.923 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:59.870+0000 7f3bff1fd700 1 -- 172.21.15.71:0/1628873906 shutdown_connections 2024-02-20T12:52:59.923 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:59.870+0000 7f3bff1fd700 1 --2- 172.21.15.71:0/1628873906 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3bf80f54b0 0x7f3bf80f58d0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:52:59.923 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:59.870+0000 7f3bff1fd700 1 -- 172.21.15.71:0/1628873906 >> 172.21.15.71:0/1628873906 conn(0x7f3bf80f0c10 msgr2=0x7f3bf80f3030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:52:59.923 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:59.870+0000 7f3bff1fd700 1 -- 172.21.15.71:0/1628873906 shutdown_connections 2024-02-20T12:52:59.923 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:59.870+0000 7f3bff1fd700 1 -- 172.21.15.71:0/1628873906 wait complete. 2024-02-20T12:52:59.923 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:59.870+0000 7f3bff1fd700 1 Processor -- start 2024-02-20T12:52:59.923 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:59.871+0000 7f3bff1fd700 1 -- start start 2024-02-20T12:52:59.923 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:59.871+0000 7f3bff1fd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3bf80f54b0 0x7f3bf81094c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:52:59.923 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:59.871+0000 7f3bff1fd700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bf8060300 con 0x7f3bf80f54b0 2024-02-20T12:52:59.923 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:59.871+0000 7f3bfcf99700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3bf80f54b0 0x7f3bf81094c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:52:59.923 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:59.871+0000 7f3bfcf99700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3bf80f54b0 0x7f3bf81094c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.71:39614/0 (socket says 172.21.15.71:39614) 2024-02-20T12:52:59.924 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:59.871+0000 7f3bfcf99700 1 -- 172.21.15.71:0/705562687 learned_addr learned my addr 172.21.15.71:0/705562687 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-02-20T12:52:59.924 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:59.871+0000 7f3bfcf99700 1 -- 172.21.15.71:0/705562687 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bec005040 con 0x7f3bf80f54b0 2024-02-20T12:52:59.924 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:59.871+0000 7f3bfcf99700 1 --2- 172.21.15.71:0/705562687 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3bf80f54b0 0x7f3bf81094c0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 rx=0x7f3bec0089f0 tx=0x7f3bec00b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:52:59.924 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:59.872+0000 7f3bf57fa700 1 -- 172.21.15.71:0/705562687 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3bec009320 con 0x7f3bf80f54b0 2024-02-20T12:52:59.924 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:59.872+0000 7f3bff1fd700 1 -- 172.21.15.71:0/705562687 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3bf8060500 con 0x7f3bf80f54b0 2024-02-20T12:52:59.924 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:59.872+0000 7f3bf57fa700 1 -- 172.21.15.71:0/705562687 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(25 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f3bec013ec0 con 0x7f3bf80f54b0 2024-02-20T12:52:59.924 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:59.872+0000 7f3bf57fa700 1 -- 172.21.15.71:0/705562687 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3bec014d30 con 0x7f3bf80f54b0 2024-02-20T12:52:59.924 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:59.872+0000 7f3bff1fd700 1 -- 172.21.15.71:0/705562687 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3bf8109b70 con 0x7f3bf80f54b0 2024-02-20T12:52:59.924 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:59.872+0000 7f3bf57fa700 1 -- 172.21.15.71:0/705562687 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 37151+0+0 (secure 0 0 0) 0x7f3bec014420 con 0x7f3bf80f54b0 2024-02-20T12:52:59.924 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:59.873+0000 7f3bff1fd700 1 -- 172.21.15.71:0/705562687 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3bf80591e0 con 0x7f3bf80f54b0 2024-02-20T12:52:59.924 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:59.873+0000 7f3bf57fa700 1 --2- 172.21.15.71:0/705562687 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f3be8031090 0x7f3be8033550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:52:59.924 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:59.873+0000 7f3bf57fa700 1 -- 172.21.15.71:0/705562687 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1065+0+0 (secure 0 0 0) 0x7f3bec0424a0 con 0x7f3bf80f54b0 2024-02-20T12:52:59.924 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:59.873+0000 7f3bf7fff700 1 --2- 172.21.15.71:0/705562687 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f3be8031090 0x7f3be8033550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:52:59.924 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:59.873+0000 7f3bf7fff700 1 --2- 172.21.15.71:0/705562687 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f3be8031090 0x7f3be8033550 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7f3be40096a0 tx=0x7f3be4006b40).ready entity=mgr.14156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:52:59.925 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:52:59.878+0000 7f3bf57fa700 1 -- 172.21.15.71:0/705562687 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+153237 (secure 0 0 0) 0x7f3bec01d9d0 con 0x7f3bf80f54b0 2024-02-20T12:53:00.087 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:00.085+0000 7f3bff1fd700 1 -- 172.21.15.71:0/705562687 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f3bf80fafa0 con 0x7f3bf80f54b0 2024-02-20T12:53:00.088 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:00.086+0000 7f3bf57fa700 1 -- 172.21.15.71:0/705562687 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7f3bec01d7f0 con 0x7f3bf80f54b0 2024-02-20T12:53:00.088 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-20T12:53:00.088 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":1,"fsid":"680ee126-cfee-11ee-95bb-87774f69a715","modified":"2024-02-20T12:49:44.699059Z","created":"2024-02-20T12:49:44.699059Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi064","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:3300","nonce":0},{"type":"v1","addr":"172.21.15.64:6789","nonce":0}]},"addr":"172.21.15.64:6789/0","public_addr":"172.21.15.64:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-20T12:53:00.089 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:00.088+0000 7f3bff1fd700 1 -- 172.21.15.71:0/705562687 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f3be8031090 msgr2=0x7f3be8033550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:53:00.089 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:00.088+0000 7f3bff1fd700 1 --2- 172.21.15.71:0/705562687 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f3be8031090 0x7f3be8033550 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7f3be40096a0 tx=0x7f3be4006b40).stop 2024-02-20T12:53:00.090 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:00.088+0000 7f3bff1fd700 1 -- 172.21.15.71:0/705562687 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3bf80f54b0 msgr2=0x7f3bf81094c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:53:00.090 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:00.088+0000 7f3bff1fd700 1 --2- 172.21.15.71:0/705562687 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3bf80f54b0 0x7f3bf81094c0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 rx=0x7f3bec0089f0 tx=0x7f3bec00b040).stop 2024-02-20T12:53:00.090 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:00.088+0000 7f3bff1fd700 1 -- 172.21.15.71:0/705562687 shutdown_connections 2024-02-20T12:53:00.090 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:00.088+0000 7f3bff1fd700 1 --2- 172.21.15.71:0/705562687 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f3be8031090 0x7f3be8033550 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:53:00.090 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:00.088+0000 7f3bff1fd700 1 --2- 172.21.15.71:0/705562687 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3bf80f54b0 0x7f3bf81094c0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:53:00.090 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:00.088+0000 7f3bff1fd700 1 -- 172.21.15.71:0/705562687 >> 172.21.15.71:0/705562687 conn(0x7f3bf80f0c10 msgr2=0x7f3bf80f1670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:53:00.090 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:00.088+0000 7f3bff1fd700 1 -- 172.21.15.71:0/705562687 shutdown_connections 2024-02-20T12:53:00.090 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:00.088+0000 7f3bff1fd700 1 -- 172.21.15.71:0/705562687 wait complete. 2024-02-20T12:53:00.090 INFO:teuthology.orchestra.run.smithi071.stderr:dumped monmap epoch 1 2024-02-20T12:53:01.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:53:01 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:52:59.721864+0000 mgr.smithi064.vriyor (mgr.14156) 65 : cluster [DBG] pgmap v57: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:53:01.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:53:01 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:53:00.087891+0000 mon.smithi064 (mon.0) 135 : audit [DBG] from='client.? 172.21.15.71:0/705562687' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-20T12:53:01.755 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-20T12:53:01.755 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/daemon-base:latest-pacific shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 680ee126-cfee-11ee-95bb-87774f69a715 -- ceph mon dump -f json 2024-02-20T12:53:03.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:53:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:53:01.722202+0000 mgr.smithi064.vriyor (mgr.14156) 66 : cluster [DBG] pgmap v58: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:53:03.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:03.584+0000 7f17abbf0700 1 -- 172.21.15.71:0/2809652771 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f17a40f6590 msgr2=0x7f17a40f69b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:53:03.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:03.584+0000 7f17abbf0700 1 --2- 172.21.15.71:0/2809652771 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f17a40f6590 0x7f17a40f69b0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 rx=0x7f17a0004760 tx=0x7f17a0009ad0).stop 2024-02-20T12:53:03.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:03.584+0000 7f17abbf0700 1 -- 172.21.15.71:0/2809652771 shutdown_connections 2024-02-20T12:53:03.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:03.584+0000 7f17abbf0700 1 --2- 172.21.15.71:0/2809652771 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f17a40f6590 0x7f17a40f69b0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:53:03.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:03.584+0000 7f17abbf0700 1 -- 172.21.15.71:0/2809652771 >> 172.21.15.71:0/2809652771 conn(0x7f17a40f1cd0 msgr2=0x7f17a40f4110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:53:03.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:03.584+0000 7f17abbf0700 1 -- 172.21.15.71:0/2809652771 shutdown_connections 2024-02-20T12:53:03.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:03.584+0000 7f17abbf0700 1 -- 172.21.15.71:0/2809652771 wait complete. 2024-02-20T12:53:03.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:03.584+0000 7f17abbf0700 1 Processor -- start 2024-02-20T12:53:03.588 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:03.585+0000 7f17abbf0700 1 -- start start 2024-02-20T12:53:03.588 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:03.585+0000 7f17abbf0700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f17a40f6590 0x7f17a41067c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:53:03.588 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:03.585+0000 7f17abbf0700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17a4106cb0 con 0x7f17a40f6590 2024-02-20T12:53:03.588 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:03.585+0000 7f17a998c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f17a40f6590 0x7f17a41067c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:53:03.588 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:03.585+0000 7f17a998c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f17a40f6590 0x7f17a41067c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.71:39636/0 (socket says 172.21.15.71:39636) 2024-02-20T12:53:03.588 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:03.585+0000 7f17a998c700 1 -- 172.21.15.71:0/1073546989 learned_addr learned my addr 172.21.15.71:0/1073546989 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-02-20T12:53:03.588 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:03.585+0000 7f17a998c700 1 -- 172.21.15.71:0/1073546989 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17a0009d70 con 0x7f17a40f6590 2024-02-20T12:53:03.588 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:03.585+0000 7f17a998c700 1 --2- 172.21.15.71:0/1073546989 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f17a40f6590 0x7f17a41067c0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 rx=0x7f17a0004170 tx=0x7f17a0009ad0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:53:03.588 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:03.585+0000 7f179a7fc700 1 -- 172.21.15.71:0/1073546989 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f17a000e040 con 0x7f17a40f6590 2024-02-20T12:53:03.588 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:03.585+0000 7f179a7fc700 1 -- 172.21.15.71:0/1073546989 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(25 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f17a0008ef0 con 0x7f17a40f6590 2024-02-20T12:53:03.588 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:03.585+0000 7f17abbf0700 1 -- 172.21.15.71:0/1073546989 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17a4106eb0 con 0x7f17a40f6590 2024-02-20T12:53:03.588 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:03.585+0000 7f179a7fc700 1 -- 172.21.15.71:0/1073546989 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f17a0014c90 con 0x7f17a40f6590 2024-02-20T12:53:03.588 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:03.586+0000 7f17abbf0700 1 -- 172.21.15.71:0/1073546989 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17a4107340 con 0x7f17a40f6590 2024-02-20T12:53:03.589 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:03.586+0000 7f179a7fc700 1 -- 172.21.15.71:0/1073546989 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 37151+0+0 (secure 0 0 0) 0x7f17a0014420 con 0x7f17a40f6590 2024-02-20T12:53:03.589 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:03.586+0000 7f17abbf0700 1 -- 172.21.15.71:0/1073546989 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17a4045bb0 con 0x7f17a40f6590 2024-02-20T12:53:03.589 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:03.586+0000 7f179a7fc700 1 --2- 172.21.15.71:0/1073546989 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f1790031090 0x7f1790033550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:53:03.590 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:03.587+0000 7f179a7fc700 1 -- 172.21.15.71:0/1073546989 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1065+0+0 (secure 0 0 0) 0x7f17a00421f0 con 0x7f17a40f6590 2024-02-20T12:53:03.590 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:03.587+0000 7f17a918b700 1 --2- 172.21.15.71:0/1073546989 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f1790031090 0x7f1790033550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:53:03.590 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:03.587+0000 7f17a918b700 1 --2- 172.21.15.71:0/1073546989 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f1790031090 0x7f1790033550 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7f17940096a0 tx=0x7f1794006b40).ready entity=mgr.14156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:53:03.593 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:03.591+0000 7f179a7fc700 1 -- 172.21.15.71:0/1073546989 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+153237 (secure 0 0 0) 0x7f17a0026300 con 0x7f17a40f6590 2024-02-20T12:53:03.799 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:03.798+0000 7f17abbf0700 1 -- 172.21.15.71:0/1073546989 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f17a41081f0 con 0x7f17a40f6590 2024-02-20T12:53:03.800 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:03.798+0000 7f179a7fc700 1 -- 172.21.15.71:0/1073546989 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7f17a0012b20 con 0x7f17a40f6590 2024-02-20T12:53:03.800 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-20T12:53:03.800 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":1,"fsid":"680ee126-cfee-11ee-95bb-87774f69a715","modified":"2024-02-20T12:49:44.699059Z","created":"2024-02-20T12:49:44.699059Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi064","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:3300","nonce":0},{"type":"v1","addr":"172.21.15.64:6789","nonce":0}]},"addr":"172.21.15.64:6789/0","public_addr":"172.21.15.64:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-20T12:53:03.801 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:03.800+0000 7f17abbf0700 1 -- 172.21.15.71:0/1073546989 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f1790031090 msgr2=0x7f1790033550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:53:03.801 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:03.800+0000 7f17abbf0700 1 --2- 172.21.15.71:0/1073546989 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f1790031090 0x7f1790033550 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7f17940096a0 tx=0x7f1794006b40).stop 2024-02-20T12:53:03.801 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:03.800+0000 7f17abbf0700 1 -- 172.21.15.71:0/1073546989 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f17a40f6590 msgr2=0x7f17a41067c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:53:03.802 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:03.800+0000 7f17abbf0700 1 --2- 172.21.15.71:0/1073546989 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f17a40f6590 0x7f17a41067c0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 rx=0x7f17a0004170 tx=0x7f17a0009ad0).stop 2024-02-20T12:53:03.802 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:03.800+0000 7f17abbf0700 1 -- 172.21.15.71:0/1073546989 shutdown_connections 2024-02-20T12:53:03.802 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:03.800+0000 7f17abbf0700 1 --2- 172.21.15.71:0/1073546989 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f1790031090 0x7f1790033550 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:53:03.802 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:03.800+0000 7f17abbf0700 1 --2- 172.21.15.71:0/1073546989 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f17a40f6590 0x7f17a41067c0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:53:03.802 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:03.800+0000 7f17abbf0700 1 -- 172.21.15.71:0/1073546989 >> 172.21.15.71:0/1073546989 conn(0x7f17a40f1cd0 msgr2=0x7f17a40f27c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:53:03.802 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:03.800+0000 7f17abbf0700 1 -- 172.21.15.71:0/1073546989 shutdown_connections 2024-02-20T12:53:03.802 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:03.800+0000 7f17abbf0700 1 -- 172.21.15.71:0/1073546989 wait complete. 2024-02-20T12:53:03.802 INFO:teuthology.orchestra.run.smithi071.stderr:dumped monmap epoch 1 2024-02-20T12:53:04.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:53:04 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:53:03.800138+0000 mon.smithi064 (mon.0) 136 : audit [DBG] from='client.? 172.21.15.71:0/1073546989' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-20T12:53:05.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:53:05 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:53:03.722555+0000 mgr.smithi064.vriyor (mgr.14156) 67 : cluster [DBG] pgmap v59: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:53:05.503 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-20T12:53:05.503 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/daemon-base:latest-pacific shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 680ee126-cfee-11ee-95bb-87774f69a715 -- ceph mon dump -f json 2024-02-20T12:53:07.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:53:07 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:53:05.722864+0000 mgr.smithi064.vriyor (mgr.14156) 68 : cluster [DBG] pgmap v60: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:53:07.316 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:07.313+0000 7fbd0156d700 1 -- 172.21.15.71:0/371020096 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbcfc0f54b0 msgr2=0x7fbcfc0f58d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:53:07.316 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:07.313+0000 7fbd0156d700 1 --2- 172.21.15.71:0/371020096 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbcfc0f54b0 0x7fbcfc0f58d0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 rx=0x7fbcec009fb0 tx=0x7fbcec009b30).stop 2024-02-20T12:53:07.316 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:07.313+0000 7fbd0156d700 1 -- 172.21.15.71:0/371020096 shutdown_connections 2024-02-20T12:53:07.316 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:07.313+0000 7fbd0156d700 1 --2- 172.21.15.71:0/371020096 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbcfc0f54b0 0x7fbcfc0f58d0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:53:07.316 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:07.313+0000 7fbd0156d700 1 -- 172.21.15.71:0/371020096 >> 172.21.15.71:0/371020096 conn(0x7fbcfc0f0c10 msgr2=0x7fbcfc0f3030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:53:07.317 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:07.313+0000 7fbd0156d700 1 -- 172.21.15.71:0/371020096 shutdown_connections 2024-02-20T12:53:07.317 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:07.313+0000 7fbd0156d700 1 -- 172.21.15.71:0/371020096 wait complete. 2024-02-20T12:53:07.317 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:07.314+0000 7fbd0156d700 1 Processor -- start 2024-02-20T12:53:07.317 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:07.314+0000 7fbd0156d700 1 -- start start 2024-02-20T12:53:07.317 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:07.314+0000 7fbd0156d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbcfc0f54b0 0x7fbcfc109510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:53:07.317 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:07.314+0000 7fbd0156d700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcfc060300 con 0x7fbcfc0f54b0 2024-02-20T12:53:07.317 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:07.314+0000 7fbcfaffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbcfc0f54b0 0x7fbcfc109510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:53:07.317 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:07.314+0000 7fbcfaffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbcfc0f54b0 0x7fbcfc109510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.71:34200/0 (socket says 172.21.15.71:34200) 2024-02-20T12:53:07.317 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:07.314+0000 7fbcfaffd700 1 -- 172.21.15.71:0/1289430041 learned_addr learned my addr 172.21.15.71:0/1289430041 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-02-20T12:53:07.317 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:07.314+0000 7fbcfaffd700 1 -- 172.21.15.71:0/1289430041 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcec005040 con 0x7fbcfc0f54b0 2024-02-20T12:53:07.317 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:07.314+0000 7fbcfaffd700 1 --2- 172.21.15.71:0/1289430041 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbcfc0f54b0 0x7fbcfc109510 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 rx=0x7fbcec0089f0 tx=0x7fbcec00b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:53:07.318 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:07.315+0000 7fbcf37fe700 1 -- 172.21.15.71:0/1289430041 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fbcec009320 con 0x7fbcfc0f54b0 2024-02-20T12:53:07.318 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:07.315+0000 7fbd0156d700 1 -- 172.21.15.71:0/1289430041 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbcfc060500 con 0x7fbcfc0f54b0 2024-02-20T12:53:07.318 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:07.315+0000 7fbcf37fe700 1 -- 172.21.15.71:0/1289430041 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(25 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7fbcec013ec0 con 0x7fbcfc0f54b0 2024-02-20T12:53:07.318 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:07.315+0000 7fbcf37fe700 1 -- 172.21.15.71:0/1289430041 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fbcec014d30 con 0x7fbcfc0f54b0 2024-02-20T12:53:07.318 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:07.315+0000 7fbd0156d700 1 -- 172.21.15.71:0/1289430041 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbcfc109bc0 con 0x7fbcfc0f54b0 2024-02-20T12:53:07.319 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:07.315+0000 7fbcf37fe700 1 -- 172.21.15.71:0/1289430041 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 37151+0+0 (secure 0 0 0) 0x7fbcec0095f0 con 0x7fbcfc0f54b0 2024-02-20T12:53:07.319 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:07.316+0000 7fbcf37fe700 1 --2- 172.21.15.71:0/1289430041 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fbce8031090 0x7fbce8033550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:53:07.319 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:07.316+0000 7fbd0156d700 1 -- 172.21.15.71:0/1289430041 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbcfc0591e0 con 0x7fbcfc0f54b0 2024-02-20T12:53:07.319 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:07.316+0000 7fbcf37fe700 1 -- 172.21.15.71:0/1289430041 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1065+0+0 (secure 0 0 0) 0x7fbcec0423e0 con 0x7fbcfc0f54b0 2024-02-20T12:53:07.319 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:07.316+0000 7fbcfa7fc700 1 --2- 172.21.15.71:0/1289430041 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fbce8031090 0x7fbce8033550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:53:07.319 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:07.316+0000 7fbcfa7fc700 1 --2- 172.21.15.71:0/1289430041 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fbce8031090 0x7fbce8033550 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7fbce40096a0 tx=0x7fbce4006b40).ready entity=mgr.14156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:53:07.322 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:07.320+0000 7fbcf37fe700 1 -- 172.21.15.71:0/1289430041 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+153237 (secure 0 0 0) 0x7fbcec01d980 con 0x7fbcfc0f54b0 2024-02-20T12:53:07.529 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:07.528+0000 7fbd0156d700 1 -- 172.21.15.71:0/1289430041 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fbcfc045bb0 con 0x7fbcfc0f54b0 2024-02-20T12:53:07.530 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:07.528+0000 7fbcf37fe700 1 -- 172.21.15.71:0/1289430041 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7fbcec01d380 con 0x7fbcfc0f54b0 2024-02-20T12:53:07.530 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-20T12:53:07.530 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":1,"fsid":"680ee126-cfee-11ee-95bb-87774f69a715","modified":"2024-02-20T12:49:44.699059Z","created":"2024-02-20T12:49:44.699059Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi064","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:3300","nonce":0},{"type":"v1","addr":"172.21.15.64:6789","nonce":0}]},"addr":"172.21.15.64:6789/0","public_addr":"172.21.15.64:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-20T12:53:07.532 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:07.530+0000 7fbd0156d700 1 -- 172.21.15.71:0/1289430041 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fbce8031090 msgr2=0x7fbce8033550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:53:07.532 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:07.530+0000 7fbd0156d700 1 --2- 172.21.15.71:0/1289430041 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fbce8031090 0x7fbce8033550 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7fbce40096a0 tx=0x7fbce4006b40).stop 2024-02-20T12:53:07.532 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:07.530+0000 7fbd0156d700 1 -- 172.21.15.71:0/1289430041 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbcfc0f54b0 msgr2=0x7fbcfc109510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:53:07.532 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:07.530+0000 7fbd0156d700 1 --2- 172.21.15.71:0/1289430041 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbcfc0f54b0 0x7fbcfc109510 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 rx=0x7fbcec0089f0 tx=0x7fbcec00b040).stop 2024-02-20T12:53:07.532 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:07.530+0000 7fbd0156d700 1 -- 172.21.15.71:0/1289430041 shutdown_connections 2024-02-20T12:53:07.532 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:07.530+0000 7fbd0156d700 1 --2- 172.21.15.71:0/1289430041 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fbce8031090 0x7fbce8033550 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:53:07.532 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:07.530+0000 7fbd0156d700 1 --2- 172.21.15.71:0/1289430041 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbcfc0f54b0 0x7fbcfc109510 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:53:07.532 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:07.530+0000 7fbd0156d700 1 -- 172.21.15.71:0/1289430041 >> 172.21.15.71:0/1289430041 conn(0x7fbcfc0f0c10 msgr2=0x7fbcfc0f16d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:53:07.532 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:07.530+0000 7fbd0156d700 1 -- 172.21.15.71:0/1289430041 shutdown_connections 2024-02-20T12:53:07.532 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:07.530+0000 7fbd0156d700 1 -- 172.21.15.71:0/1289430041 wait complete. 2024-02-20T12:53:07.533 INFO:teuthology.orchestra.run.smithi071.stderr:dumped monmap epoch 1 2024-02-20T12:53:08.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:53:08 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:53:07.530188+0000 mon.smithi064 (mon.0) 137 : audit [DBG] from='client.? 172.21.15.71:0/1289430041' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-20T12:53:09.143 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-20T12:53:09.143 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/daemon-base:latest-pacific shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 680ee126-cfee-11ee-95bb-87774f69a715 -- ceph mon dump -f json 2024-02-20T12:53:09.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:53:09 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:53:07.723170+0000 mgr.smithi064.vriyor (mgr.14156) 69 : cluster [DBG] pgmap v61: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:53:11.016 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:11.014+0000 7f52ae834700 1 -- 172.21.15.71:0/277897663 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f52a80f52d0 msgr2=0x7f52a80f56f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:53:11.016 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:11.014+0000 7f52ae834700 1 --2- 172.21.15.71:0/277897663 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f52a80f52d0 0x7f52a80f56f0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 rx=0x7f5298009fb0 tx=0x7f5298009b30).stop 2024-02-20T12:53:11.016 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:11.014+0000 7f52ae834700 1 -- 172.21.15.71:0/277897663 shutdown_connections 2024-02-20T12:53:11.016 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:11.014+0000 7f52ae834700 1 --2- 172.21.15.71:0/277897663 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f52a80f52d0 0x7f52a80f56f0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:53:11.017 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:11.014+0000 7f52ae834700 1 -- 172.21.15.71:0/277897663 >> 172.21.15.71:0/277897663 conn(0x7f52a80f0a30 msgr2=0x7f52a80f2e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:53:11.017 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:11.015+0000 7f52ae834700 1 -- 172.21.15.71:0/277897663 shutdown_connections 2024-02-20T12:53:11.017 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:11.015+0000 7f52ae834700 1 -- 172.21.15.71:0/277897663 wait complete. 2024-02-20T12:53:11.018 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:11.015+0000 7f52ae834700 1 Processor -- start 2024-02-20T12:53:11.018 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:11.015+0000 7f52ae834700 1 -- start start 2024-02-20T12:53:11.018 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:11.015+0000 7f52ae834700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f52a80f52d0 0x7f52a81065f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:53:11.018 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:11.015+0000 7f52ae834700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52a8106ae0 con 0x7f52a80f52d0 2024-02-20T12:53:11.018 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:11.016+0000 7f52ad832700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f52a80f52d0 0x7f52a81065f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:53:11.018 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:11.016+0000 7f52ad832700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f52a80f52d0 0x7f52a81065f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.71:34214/0 (socket says 172.21.15.71:34214) 2024-02-20T12:53:11.018 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:11.016+0000 7f52ad832700 1 -- 172.21.15.71:0/784889335 learned_addr learned my addr 172.21.15.71:0/784889335 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-02-20T12:53:11.018 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:11.016+0000 7f52ad832700 1 -- 172.21.15.71:0/784889335 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5298005040 con 0x7f52a80f52d0 2024-02-20T12:53:11.018 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:11.016+0000 7f52ad832700 1 --2- 172.21.15.71:0/784889335 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f52a80f52d0 0x7f52a81065f0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 rx=0x7f52980089f0 tx=0x7f529800b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:53:11.019 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:11.017+0000 7f529e7fc700 1 -- 172.21.15.71:0/784889335 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5298009320 con 0x7f52a80f52d0 2024-02-20T12:53:11.020 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:11.017+0000 7f52ae834700 1 -- 172.21.15.71:0/784889335 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52a8106ce0 con 0x7f52a80f52d0 2024-02-20T12:53:11.020 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:11.017+0000 7f529e7fc700 1 -- 172.21.15.71:0/784889335 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(25 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f5298013ec0 con 0x7f52a80f52d0 2024-02-20T12:53:11.020 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:11.017+0000 7f529e7fc700 1 -- 172.21.15.71:0/784889335 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5298014d30 con 0x7f52a80f52d0 2024-02-20T12:53:11.020 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:11.017+0000 7f52ae834700 1 -- 172.21.15.71:0/784889335 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52a8107170 con 0x7f52a80f52d0 2024-02-20T12:53:11.020 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:11.018+0000 7f529e7fc700 1 -- 172.21.15.71:0/784889335 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 37151+0+0 (secure 0 0 0) 0x7f52980263a0 con 0x7f52a80f52d0 2024-02-20T12:53:11.020 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:11.018+0000 7f52ae834700 1 -- 172.21.15.71:0/784889335 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52a8045bb0 con 0x7f52a80f52d0 2024-02-20T12:53:11.020 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:11.018+0000 7f529e7fc700 1 --2- 172.21.15.71:0/784889335 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f5294031090 0x7f5294033550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:53:11.020 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:11.018+0000 7f529e7fc700 1 -- 172.21.15.71:0/784889335 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1065+0+0 (secure 0 0 0) 0x7f52980424f0 con 0x7f52a80f52d0 2024-02-20T12:53:11.020 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:11.018+0000 7f52ad031700 1 --2- 172.21.15.71:0/784889335 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f5294031090 0x7f5294033550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:53:11.020 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:11.019+0000 7f52ad031700 1 --2- 172.21.15.71:0/784889335 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f5294031090 0x7f5294033550 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7f52a40096a0 tx=0x7f52a4006b40).ready entity=mgr.14156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:53:11.024 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:11.024+0000 7f529e7fc700 1 -- 172.21.15.71:0/784889335 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+153237 (secure 0 0 0) 0x7f5298012330 con 0x7f52a80f52d0 2024-02-20T12:53:11.231 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:11.230+0000 7f52ae834700 1 -- 172.21.15.71:0/784889335 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f52a8108020 con 0x7f52a80f52d0 2024-02-20T12:53:11.232 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:11.231+0000 7f529e7fc700 1 -- 172.21.15.71:0/784889335 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7f5298012330 con 0x7f52a80f52d0 2024-02-20T12:53:11.233 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-20T12:53:11.233 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":1,"fsid":"680ee126-cfee-11ee-95bb-87774f69a715","modified":"2024-02-20T12:49:44.699059Z","created":"2024-02-20T12:49:44.699059Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi064","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:3300","nonce":0},{"type":"v1","addr":"172.21.15.64:6789","nonce":0}]},"addr":"172.21.15.64:6789/0","public_addr":"172.21.15.64:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-20T12:53:11.234 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:11.232+0000 7f52ae834700 1 -- 172.21.15.71:0/784889335 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f5294031090 msgr2=0x7f5294033550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:53:11.234 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:11.232+0000 7f52ae834700 1 --2- 172.21.15.71:0/784889335 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f5294031090 0x7f5294033550 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7f52a40096a0 tx=0x7f52a4006b40).stop 2024-02-20T12:53:11.234 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:11.232+0000 7f52ae834700 1 -- 172.21.15.71:0/784889335 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f52a80f52d0 msgr2=0x7f52a81065f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:53:11.234 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:11.232+0000 7f52ae834700 1 --2- 172.21.15.71:0/784889335 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f52a80f52d0 0x7f52a81065f0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 rx=0x7f52980089f0 tx=0x7f529800b040).stop 2024-02-20T12:53:11.235 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:11.232+0000 7f52ae834700 1 -- 172.21.15.71:0/784889335 shutdown_connections 2024-02-20T12:53:11.235 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:11.232+0000 7f52ae834700 1 --2- 172.21.15.71:0/784889335 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f5294031090 0x7f5294033550 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:53:11.235 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:11.232+0000 7f52ae834700 1 --2- 172.21.15.71:0/784889335 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f52a80f52d0 0x7f52a81065f0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:53:11.235 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:11.232+0000 7f52ae834700 1 -- 172.21.15.71:0/784889335 >> 172.21.15.71:0/784889335 conn(0x7f52a80f0a30 msgr2=0x7f52a80f1580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:53:11.235 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:11.232+0000 7f52ae834700 1 -- 172.21.15.71:0/784889335 shutdown_connections 2024-02-20T12:53:11.235 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:11.232+0000 7f52ae834700 1 -- 172.21.15.71:0/784889335 wait complete. 2024-02-20T12:53:11.236 INFO:teuthology.orchestra.run.smithi071.stderr:dumped monmap epoch 1 2024-02-20T12:53:11.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:53:11 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:53:09.723416+0000 mgr.smithi064.vriyor (mgr.14156) 70 : cluster [DBG] pgmap v62: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:53:12.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:53:12 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:53:11.232340+0000 mon.smithi064 (mon.0) 138 : audit [DBG] from='client.? 172.21.15.71:0/784889335' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-20T12:53:12.776 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-20T12:53:12.776 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/daemon-base:latest-pacific shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 680ee126-cfee-11ee-95bb-87774f69a715 -- ceph mon dump -f json 2024-02-20T12:53:13.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:53:13 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:53:11.723745+0000 mgr.smithi064.vriyor (mgr.14156) 71 : cluster [DBG] pgmap v63: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:53:14.769 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:14.767+0000 7ff20c3e5700 1 -- 172.21.15.71:0/3902997871 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff2040f5480 msgr2=0x7ff2040f58a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:53:14.769 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:14.767+0000 7ff20c3e5700 1 --2- 172.21.15.71:0/3902997871 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff2040f5480 0x7ff2040f58a0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 rx=0x7ff1f8009fb0 tx=0x7ff1f8009b30).stop 2024-02-20T12:53:14.769 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:14.767+0000 7ff20c3e5700 1 -- 172.21.15.71:0/3902997871 shutdown_connections 2024-02-20T12:53:14.769 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:14.767+0000 7ff20c3e5700 1 --2- 172.21.15.71:0/3902997871 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff2040f5480 0x7ff2040f58a0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:53:14.769 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:14.767+0000 7ff20c3e5700 1 -- 172.21.15.71:0/3902997871 >> 172.21.15.71:0/3902997871 conn(0x7ff2040f0be0 msgr2=0x7ff2040f3000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:53:14.769 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:14.767+0000 7ff20c3e5700 1 -- 172.21.15.71:0/3902997871 shutdown_connections 2024-02-20T12:53:14.769 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:14.767+0000 7ff20c3e5700 1 -- 172.21.15.71:0/3902997871 wait complete. 2024-02-20T12:53:14.769 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:14.768+0000 7ff20c3e5700 1 Processor -- start 2024-02-20T12:53:14.770 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:14.768+0000 7ff20c3e5700 1 -- start start 2024-02-20T12:53:14.770 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:14.768+0000 7ff20c3e5700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff2040f5480 0x7ff2041056f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:53:14.770 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:14.768+0000 7ff20c3e5700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff204105be0 con 0x7ff2040f5480 2024-02-20T12:53:14.770 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:14.768+0000 7ff20a181700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff2040f5480 0x7ff2041056f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:53:14.770 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:14.768+0000 7ff20a181700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff2040f5480 0x7ff2041056f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.71:34232/0 (socket says 172.21.15.71:34232) 2024-02-20T12:53:14.770 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:14.768+0000 7ff20a181700 1 -- 172.21.15.71:0/3385868255 learned_addr learned my addr 172.21.15.71:0/3385868255 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-02-20T12:53:14.770 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:14.769+0000 7ff20a181700 1 -- 172.21.15.71:0/3385868255 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1f8005040 con 0x7ff2040f5480 2024-02-20T12:53:14.770 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:14.769+0000 7ff20a181700 1 --2- 172.21.15.71:0/3385868255 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff2040f5480 0x7ff2041056f0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 rx=0x7ff1f8000c00 tx=0x7ff1f800b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:53:14.771 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:14.769+0000 7ff1f6ffd700 1 -- 172.21.15.71:0/3385868255 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff1f80095f0 con 0x7ff2040f5480 2024-02-20T12:53:14.771 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:14.769+0000 7ff20c3e5700 1 -- 172.21.15.71:0/3385868255 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff204105de0 con 0x7ff2040f5480 2024-02-20T12:53:14.771 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:14.769+0000 7ff1f6ffd700 1 -- 172.21.15.71:0/3385868255 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(25 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7ff1f8007e50 con 0x7ff2040f5480 2024-02-20T12:53:14.771 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:14.769+0000 7ff1f6ffd700 1 -- 172.21.15.71:0/3385868255 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff1f8014e70 con 0x7ff2040f5480 2024-02-20T12:53:14.772 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:14.769+0000 7ff20c3e5700 1 -- 172.21.15.71:0/3385868255 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff204106270 con 0x7ff2040f5480 2024-02-20T12:53:14.772 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:14.770+0000 7ff1f6ffd700 1 -- 172.21.15.71:0/3385868255 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 37151+0+0 (secure 0 0 0) 0x7ff1f8025070 con 0x7ff2040f5480 2024-02-20T12:53:14.772 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:14.770+0000 7ff1f6ffd700 1 --2- 172.21.15.71:0/3385868255 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7ff1f0031090 0x7ff1f0033550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:53:14.772 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:14.770+0000 7ff20c3e5700 1 -- 172.21.15.71:0/3385868255 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff204045bb0 con 0x7ff2040f5480 2024-02-20T12:53:14.772 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:14.770+0000 7ff1f6ffd700 1 -- 172.21.15.71:0/3385868255 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1065+0+0 (secure 0 0 0) 0x7ff1f8041d60 con 0x7ff2040f5480 2024-02-20T12:53:14.772 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:14.770+0000 7ff209980700 1 --2- 172.21.15.71:0/3385868255 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7ff1f0031090 0x7ff1f0033550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:53:14.772 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:14.770+0000 7ff209980700 1 --2- 172.21.15.71:0/3385868255 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7ff1f0031090 0x7ff1f0033550 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7ff2000096a0 tx=0x7ff200006b40).ready entity=mgr.14156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:53:14.775 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:14.775+0000 7ff1f6ffd700 1 -- 172.21.15.71:0/3385868255 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+153237 (secure 0 0 0) 0x7ff1f8045070 con 0x7ff2040f5480 2024-02-20T12:53:14.982 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:14.982+0000 7ff20c3e5700 1 -- 172.21.15.71:0/3385868255 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7ff2040fb060 con 0x7ff2040f5480 2024-02-20T12:53:14.983 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:14.982+0000 7ff1f6ffd700 1 -- 172.21.15.71:0/3385868255 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7ff1f800f4e0 con 0x7ff2040f5480 2024-02-20T12:53:14.983 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-20T12:53:14.983 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":1,"fsid":"680ee126-cfee-11ee-95bb-87774f69a715","modified":"2024-02-20T12:49:44.699059Z","created":"2024-02-20T12:49:44.699059Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi064","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:3300","nonce":0},{"type":"v1","addr":"172.21.15.64:6789","nonce":0}]},"addr":"172.21.15.64:6789/0","public_addr":"172.21.15.64:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-20T12:53:14.985 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:14.984+0000 7ff20c3e5700 1 -- 172.21.15.71:0/3385868255 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7ff1f0031090 msgr2=0x7ff1f0033550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:53:14.985 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:14.984+0000 7ff20c3e5700 1 --2- 172.21.15.71:0/3385868255 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7ff1f0031090 0x7ff1f0033550 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7ff2000096a0 tx=0x7ff200006b40).stop 2024-02-20T12:53:14.985 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:14.984+0000 7ff20c3e5700 1 -- 172.21.15.71:0/3385868255 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff2040f5480 msgr2=0x7ff2041056f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:53:14.985 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:14.984+0000 7ff20c3e5700 1 --2- 172.21.15.71:0/3385868255 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff2040f5480 0x7ff2041056f0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 rx=0x7ff1f8000c00 tx=0x7ff1f800b040).stop 2024-02-20T12:53:14.985 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:14.984+0000 7ff20c3e5700 1 -- 172.21.15.71:0/3385868255 shutdown_connections 2024-02-20T12:53:14.985 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:14.984+0000 7ff20c3e5700 1 --2- 172.21.15.71:0/3385868255 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7ff1f0031090 0x7ff1f0033550 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:53:14.985 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:14.984+0000 7ff20c3e5700 1 --2- 172.21.15.71:0/3385868255 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff2040f5480 0x7ff2041056f0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:53:14.985 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:14.984+0000 7ff20c3e5700 1 -- 172.21.15.71:0/3385868255 >> 172.21.15.71:0/3385868255 conn(0x7ff2040f0be0 msgr2=0x7ff2040f16d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:53:14.985 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:14.984+0000 7ff20c3e5700 1 -- 172.21.15.71:0/3385868255 shutdown_connections 2024-02-20T12:53:14.985 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:14.984+0000 7ff20c3e5700 1 -- 172.21.15.71:0/3385868255 wait complete. 2024-02-20T12:53:14.986 INFO:teuthology.orchestra.run.smithi071.stderr:dumped monmap epoch 1 2024-02-20T12:53:15.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:53:15 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:53:13.724129+0000 mgr.smithi064.vriyor (mgr.14156) 72 : cluster [DBG] pgmap v64: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:53:15.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:53:15 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:53:14.983317+0000 mon.smithi064 (mon.0) 139 : audit [DBG] from='client.? 172.21.15.71:0/3385868255' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-20T12:53:16.670 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-20T12:53:16.670 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/daemon-base:latest-pacific shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 680ee126-cfee-11ee-95bb-87774f69a715 -- ceph mon dump -f json 2024-02-20T12:53:17.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:53:17 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:53:15.724536+0000 mgr.smithi064.vriyor (mgr.14156) 73 : cluster [DBG] pgmap v65: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:53:18.451 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:18.425+0000 7f6cdce3c700 1 -- 172.21.15.71:0/3649904757 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6cd80f5480 msgr2=0x7f6cd80f58a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:53:18.451 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:18.425+0000 7f6cdce3c700 1 --2- 172.21.15.71:0/3649904757 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6cd80f5480 0x7f6cd80f58a0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 rx=0x7f6cc8009fb0 tx=0x7f6cc8009b30).stop 2024-02-20T12:53:18.451 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:18.425+0000 7f6cdce3c700 1 -- 172.21.15.71:0/3649904757 shutdown_connections 2024-02-20T12:53:18.451 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:18.425+0000 7f6cdce3c700 1 --2- 172.21.15.71:0/3649904757 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6cd80f5480 0x7f6cd80f58a0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:53:18.451 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:18.425+0000 7f6cdce3c700 1 -- 172.21.15.71:0/3649904757 >> 172.21.15.71:0/3649904757 conn(0x7f6cd80f0be0 msgr2=0x7f6cd80f3000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:53:18.451 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:18.425+0000 7f6cdce3c700 1 -- 172.21.15.71:0/3649904757 shutdown_connections 2024-02-20T12:53:18.452 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:18.425+0000 7f6cdce3c700 1 -- 172.21.15.71:0/3649904757 wait complete. 2024-02-20T12:53:18.452 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:18.425+0000 7f6cdce3c700 1 Processor -- start 2024-02-20T12:53:18.452 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:18.425+0000 7f6cdce3c700 1 -- start start 2024-02-20T12:53:18.452 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:18.426+0000 7f6cdce3c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6cd80f5480 0x7f6cd81089b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:53:18.452 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:18.426+0000 7f6cdce3c700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cd8108ea0 con 0x7f6cd80f5480 2024-02-20T12:53:18.452 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:18.426+0000 7f6cd659c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6cd80f5480 0x7f6cd81089b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:53:18.452 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:18.426+0000 7f6cd659c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6cd80f5480 0x7f6cd81089b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.71:51038/0 (socket says 172.21.15.71:51038) 2024-02-20T12:53:18.452 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:18.426+0000 7f6cd659c700 1 -- 172.21.15.71:0/1250932668 learned_addr learned my addr 172.21.15.71:0/1250932668 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-02-20T12:53:18.452 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:18.426+0000 7f6cd659c700 1 -- 172.21.15.71:0/1250932668 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cc8005040 con 0x7f6cd80f5480 2024-02-20T12:53:18.452 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:18.426+0000 7f6cd659c700 1 --2- 172.21.15.71:0/1250932668 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6cd80f5480 0x7f6cd81089b0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 rx=0x7f6cc80089f0 tx=0x7f6cc800b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:53:18.452 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:18.427+0000 7f6cc6ffd700 1 -- 172.21.15.71:0/1250932668 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6cc8009320 con 0x7f6cd80f5480 2024-02-20T12:53:18.452 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:18.427+0000 7f6cdce3c700 1 -- 172.21.15.71:0/1250932668 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6cd81090a0 con 0x7f6cd80f5480 2024-02-20T12:53:18.452 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:18.427+0000 7f6cc6ffd700 1 -- 172.21.15.71:0/1250932668 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(25 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f6cc8013ec0 con 0x7f6cd80f5480 2024-02-20T12:53:18.452 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:18.427+0000 7f6cc6ffd700 1 -- 172.21.15.71:0/1250932668 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6cc8014d30 con 0x7f6cd80f5480 2024-02-20T12:53:18.453 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:18.427+0000 7f6cdce3c700 1 -- 172.21.15.71:0/1250932668 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6cd8109530 con 0x7f6cd80f5480 2024-02-20T12:53:18.453 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:18.427+0000 7f6cc6ffd700 1 -- 172.21.15.71:0/1250932668 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 37151+0+0 (secure 0 0 0) 0x7f6cc8014420 con 0x7f6cd80f5480 2024-02-20T12:53:18.453 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:18.427+0000 7f6cc6ffd700 1 --2- 172.21.15.71:0/1250932668 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f6cc0031090 0x7f6cc0033550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:53:18.453 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:18.427+0000 7f6cdce3c700 1 -- 172.21.15.71:0/1250932668 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6cd8045bb0 con 0x7f6cd80f5480 2024-02-20T12:53:18.453 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:18.428+0000 7f6cc6ffd700 1 -- 172.21.15.71:0/1250932668 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1065+0+0 (secure 0 0 0) 0x7f6cc80423e0 con 0x7f6cd80f5480 2024-02-20T12:53:18.453 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:18.428+0000 7f6cd5d9b700 1 --2- 172.21.15.71:0/1250932668 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f6cc0031090 0x7f6cc0033550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:53:18.453 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:18.428+0000 7f6cd5d9b700 1 --2- 172.21.15.71:0/1250932668 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f6cc0031090 0x7f6cc0033550 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7f6ccc0096a0 tx=0x7f6ccc006b40).ready entity=mgr.14156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:53:18.453 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:18.432+0000 7f6cc6ffd700 1 -- 172.21.15.71:0/1250932668 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+153237 (secure 0 0 0) 0x7f6cc8014750 con 0x7f6cd80f5480 2024-02-20T12:53:18.642 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:18.641+0000 7f6cdce3c700 1 -- 172.21.15.71:0/1250932668 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f6cd80fb060 con 0x7f6cd80f5480 2024-02-20T12:53:18.642 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:18.642+0000 7f6cc6ffd700 1 -- 172.21.15.71:0/1250932668 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7f6cc801d7f0 con 0x7f6cd80f5480 2024-02-20T12:53:18.642 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-20T12:53:18.643 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":1,"fsid":"680ee126-cfee-11ee-95bb-87774f69a715","modified":"2024-02-20T12:49:44.699059Z","created":"2024-02-20T12:49:44.699059Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi064","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:3300","nonce":0},{"type":"v1","addr":"172.21.15.64:6789","nonce":0}]},"addr":"172.21.15.64:6789/0","public_addr":"172.21.15.64:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-20T12:53:18.644 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:18.643+0000 7f6cdce3c700 1 -- 172.21.15.71:0/1250932668 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f6cc0031090 msgr2=0x7f6cc0033550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:53:18.644 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:18.643+0000 7f6cdce3c700 1 --2- 172.21.15.71:0/1250932668 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f6cc0031090 0x7f6cc0033550 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7f6ccc0096a0 tx=0x7f6ccc006b40).stop 2024-02-20T12:53:18.644 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:18.643+0000 7f6cdce3c700 1 -- 172.21.15.71:0/1250932668 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6cd80f5480 msgr2=0x7f6cd81089b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:53:18.644 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:18.643+0000 7f6cdce3c700 1 --2- 172.21.15.71:0/1250932668 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6cd80f5480 0x7f6cd81089b0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 rx=0x7f6cc80089f0 tx=0x7f6cc800b040).stop 2024-02-20T12:53:18.644 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:18.644+0000 7f6cdce3c700 1 -- 172.21.15.71:0/1250932668 shutdown_connections 2024-02-20T12:53:18.645 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:18.644+0000 7f6cdce3c700 1 --2- 172.21.15.71:0/1250932668 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f6cc0031090 0x7f6cc0033550 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:53:18.645 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:18.644+0000 7f6cdce3c700 1 --2- 172.21.15.71:0/1250932668 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6cd80f5480 0x7f6cd81089b0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:53:18.645 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:18.644+0000 7f6cdce3c700 1 -- 172.21.15.71:0/1250932668 >> 172.21.15.71:0/1250932668 conn(0x7f6cd80f0be0 msgr2=0x7f6cd805fee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:53:18.645 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:18.644+0000 7f6cdce3c700 1 -- 172.21.15.71:0/1250932668 shutdown_connections 2024-02-20T12:53:18.645 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:18.644+0000 7f6cdce3c700 1 -- 172.21.15.71:0/1250932668 wait complete. 2024-02-20T12:53:18.645 INFO:teuthology.orchestra.run.smithi071.stderr:dumped monmap epoch 1 2024-02-20T12:53:19.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:53:19 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:53:17.724856+0000 mgr.smithi064.vriyor (mgr.14156) 74 : cluster [DBG] pgmap v66: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:53:19.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:53:19 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:53:18.642629+0000 mon.smithi064 (mon.0) 140 : audit [DBG] from='client.? 172.21.15.71:0/1250932668' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-20T12:53:20.514 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-20T12:53:20.514 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/daemon-base:latest-pacific shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 680ee126-cfee-11ee-95bb-87774f69a715 -- ceph mon dump -f json 2024-02-20T12:53:21.315 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:53:21 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:53:19.725206+0000 mgr.smithi064.vriyor (mgr.14156) 75 : cluster [DBG] pgmap v67: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:53:22.426 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:22.423+0000 7f705abc0700 1 -- 172.21.15.71:0/4093228953 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7054060300 msgr2=0x7f70540f91c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:53:22.426 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:22.423+0000 7f705abc0700 1 --2- 172.21.15.71:0/4093228953 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7054060300 0x7f70540f91c0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 rx=0x7f7048009fb0 tx=0x7f7048009b30).stop 2024-02-20T12:53:22.426 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:22.423+0000 7f705abc0700 1 -- 172.21.15.71:0/4093228953 shutdown_connections 2024-02-20T12:53:22.426 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:22.423+0000 7f705abc0700 1 --2- 172.21.15.71:0/4093228953 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7054060300 0x7f70540f91c0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:53:22.426 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:22.423+0000 7f705abc0700 1 -- 172.21.15.71:0/4093228953 >> 172.21.15.71:0/4093228953 conn(0x7f70540f0be0 msgr2=0x7f70540f3000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:53:22.427 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:22.423+0000 7f705abc0700 1 -- 172.21.15.71:0/4093228953 shutdown_connections 2024-02-20T12:53:22.427 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:22.423+0000 7f705abc0700 1 -- 172.21.15.71:0/4093228953 wait complete. 2024-02-20T12:53:22.427 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:22.423+0000 7f705abc0700 1 Processor -- start 2024-02-20T12:53:22.427 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:22.423+0000 7f705abc0700 1 -- start start 2024-02-20T12:53:22.427 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:22.424+0000 7f705abc0700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7054060300 0x7f7054106780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:53:22.427 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:22.424+0000 7f705abc0700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7054106c70 con 0x7f7054060300 2024-02-20T12:53:22.427 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:22.424+0000 7f705895c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7054060300 0x7f7054106780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:53:22.427 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:22.424+0000 7f705895c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7054060300 0x7f7054106780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.71:51064/0 (socket says 172.21.15.71:51064) 2024-02-20T12:53:22.427 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:22.424+0000 7f705895c700 1 -- 172.21.15.71:0/3302711688 learned_addr learned my addr 172.21.15.71:0/3302711688 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-02-20T12:53:22.427 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:22.424+0000 7f705895c700 1 -- 172.21.15.71:0/3302711688 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7048005040 con 0x7f7054060300 2024-02-20T12:53:22.427 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:22.424+0000 7f705895c700 1 --2- 172.21.15.71:0/3302711688 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7054060300 0x7f7054106780 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 rx=0x7f7048009f80 tx=0x7f704800b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:53:22.427 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:22.424+0000 7f70517fa700 1 -- 172.21.15.71:0/3302711688 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7048009320 con 0x7f7054060300 2024-02-20T12:53:22.428 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:22.425+0000 7f705abc0700 1 -- 172.21.15.71:0/3302711688 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7054106e70 con 0x7f7054060300 2024-02-20T12:53:22.428 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:22.425+0000 7f70517fa700 1 -- 172.21.15.71:0/3302711688 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(25 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f7048013680 con 0x7f7054060300 2024-02-20T12:53:22.428 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:22.425+0000 7f70517fa700 1 -- 172.21.15.71:0/3302711688 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f704801eda0 con 0x7f7054060300 2024-02-20T12:53:22.428 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:22.425+0000 7f705abc0700 1 -- 172.21.15.71:0/3302711688 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7054107300 con 0x7f7054060300 2024-02-20T12:53:22.428 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:22.425+0000 7f70517fa700 1 -- 172.21.15.71:0/3302711688 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 37151+0+0 (secure 0 0 0) 0x7f704801e420 con 0x7f7054060300 2024-02-20T12:53:22.428 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:22.426+0000 7f705abc0700 1 -- 172.21.15.71:0/3302711688 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7054045bb0 con 0x7f7054060300 2024-02-20T12:53:22.428 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:22.426+0000 7f70517fa700 1 --2- 172.21.15.71:0/3302711688 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f70440310e0 0x7f70440335a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:53:22.428 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:22.426+0000 7f70517fa700 1 -- 172.21.15.71:0/3302711688 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1065+0+0 (secure 0 0 0) 0x7f7048042410 con 0x7f7054060300 2024-02-20T12:53:22.429 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:22.426+0000 7f7053fff700 1 --2- 172.21.15.71:0/3302711688 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f70440310e0 0x7f70440335a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:53:22.429 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:22.426+0000 7f7053fff700 1 --2- 172.21.15.71:0/3302711688 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f70440310e0 0x7f70440335a0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7f70400096a0 tx=0x7f7040006b40).ready entity=mgr.14156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:53:22.431 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:22.431+0000 7f70517fa700 1 -- 172.21.15.71:0/3302711688 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+153237 (secure 0 0 0) 0x7f704801cb10 con 0x7f7054060300 2024-02-20T12:53:22.637 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:22.637+0000 7f705abc0700 1 -- 172.21.15.71:0/3302711688 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f70541081b0 con 0x7f7054060300 2024-02-20T12:53:22.638 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:22.637+0000 7f70517fa700 1 -- 172.21.15.71:0/3302711688 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7f704802c2d0 con 0x7f7054060300 2024-02-20T12:53:22.638 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-20T12:53:22.638 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":1,"fsid":"680ee126-cfee-11ee-95bb-87774f69a715","modified":"2024-02-20T12:49:44.699059Z","created":"2024-02-20T12:49:44.699059Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi064","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:3300","nonce":0},{"type":"v1","addr":"172.21.15.64:6789","nonce":0}]},"addr":"172.21.15.64:6789/0","public_addr":"172.21.15.64:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-20T12:53:22.640 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:22.639+0000 7f705abc0700 1 -- 172.21.15.71:0/3302711688 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f70440310e0 msgr2=0x7f70440335a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:53:22.640 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:22.639+0000 7f705abc0700 1 --2- 172.21.15.71:0/3302711688 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f70440310e0 0x7f70440335a0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7f70400096a0 tx=0x7f7040006b40).stop 2024-02-20T12:53:22.640 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:22.639+0000 7f705abc0700 1 -- 172.21.15.71:0/3302711688 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7054060300 msgr2=0x7f7054106780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:53:22.640 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:22.639+0000 7f705abc0700 1 --2- 172.21.15.71:0/3302711688 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7054060300 0x7f7054106780 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 rx=0x7f7048009f80 tx=0x7f704800b040).stop 2024-02-20T12:53:22.640 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:22.639+0000 7f705abc0700 1 -- 172.21.15.71:0/3302711688 shutdown_connections 2024-02-20T12:53:22.640 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:22.639+0000 7f705abc0700 1 --2- 172.21.15.71:0/3302711688 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f70440310e0 0x7f70440335a0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:53:22.640 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:22.639+0000 7f705abc0700 1 --2- 172.21.15.71:0/3302711688 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7054060300 0x7f7054106780 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:53:22.640 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:22.639+0000 7f705abc0700 1 -- 172.21.15.71:0/3302711688 >> 172.21.15.71:0/3302711688 conn(0x7f70540f0be0 msgr2=0x7f70540f1720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:53:22.640 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:22.639+0000 7f705abc0700 1 -- 172.21.15.71:0/3302711688 shutdown_connections 2024-02-20T12:53:22.640 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:22.639+0000 7f705abc0700 1 -- 172.21.15.71:0/3302711688 wait complete. 2024-02-20T12:53:22.641 INFO:teuthology.orchestra.run.smithi071.stderr:dumped monmap epoch 1 2024-02-20T12:53:23.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:53:23 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:53:21.725572+0000 mgr.smithi064.vriyor (mgr.14156) 76 : cluster [DBG] pgmap v68: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:53:23.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:53:23 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:53:22.638269+0000 mon.smithi064 (mon.0) 141 : audit [DBG] from='client.? 172.21.15.71:0/3302711688' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-20T12:53:24.224 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-20T12:53:24.224 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/daemon-base:latest-pacific shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 680ee126-cfee-11ee-95bb-87774f69a715 -- ceph mon dump -f json 2024-02-20T12:53:25.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:53:25 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:53:23.725878+0000 mgr.smithi064.vriyor (mgr.14156) 77 : cluster [DBG] pgmap v69: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:53:26.118 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:26.106+0000 7efc6f727700 1 -- 172.21.15.71:0/234385665 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efc680f5480 msgr2=0x7efc680f58a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:53:26.118 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:26.106+0000 7efc6f727700 1 --2- 172.21.15.71:0/234385665 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efc680f5480 0x7efc680f58a0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 rx=0x7efc64009fb0 tx=0x7efc64009b30).stop 2024-02-20T12:53:26.118 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:26.106+0000 7efc6f727700 1 -- 172.21.15.71:0/234385665 shutdown_connections 2024-02-20T12:53:26.118 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:26.106+0000 7efc6f727700 1 --2- 172.21.15.71:0/234385665 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efc680f5480 0x7efc680f58a0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:53:26.118 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:26.106+0000 7efc6f727700 1 -- 172.21.15.71:0/234385665 >> 172.21.15.71:0/234385665 conn(0x7efc680f0be0 msgr2=0x7efc680f3000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:53:26.118 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:26.106+0000 7efc6f727700 1 -- 172.21.15.71:0/234385665 shutdown_connections 2024-02-20T12:53:26.118 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:26.106+0000 7efc6f727700 1 -- 172.21.15.71:0/234385665 wait complete. 2024-02-20T12:53:26.118 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:26.107+0000 7efc6f727700 1 Processor -- start 2024-02-20T12:53:26.119 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:26.107+0000 7efc6f727700 1 -- start start 2024-02-20T12:53:26.119 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:26.107+0000 7efc6f727700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efc680f5480 0x7efc681056f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:53:26.119 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:26.107+0000 7efc6f727700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc68105be0 con 0x7efc680f5480 2024-02-20T12:53:26.119 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:26.107+0000 7efc6d4c3700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efc680f5480 0x7efc681056f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:53:26.119 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:26.107+0000 7efc6d4c3700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efc680f5480 0x7efc681056f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.71:51080/0 (socket says 172.21.15.71:51080) 2024-02-20T12:53:26.119 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:26.107+0000 7efc6d4c3700 1 -- 172.21.15.71:0/3502513341 learned_addr learned my addr 172.21.15.71:0/3502513341 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-02-20T12:53:26.119 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:26.107+0000 7efc6d4c3700 1 -- 172.21.15.71:0/3502513341 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc64005040 con 0x7efc680f5480 2024-02-20T12:53:26.119 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:26.108+0000 7efc6d4c3700 1 --2- 172.21.15.71:0/3502513341 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efc680f5480 0x7efc681056f0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 rx=0x7efc640089f0 tx=0x7efc6400b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:53:26.119 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:26.108+0000 7efc5dffb700 1 -- 172.21.15.71:0/3502513341 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7efc64009320 con 0x7efc680f5480 2024-02-20T12:53:26.119 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:26.108+0000 7efc6f727700 1 -- 172.21.15.71:0/3502513341 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc68105de0 con 0x7efc680f5480 2024-02-20T12:53:26.119 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:26.108+0000 7efc5dffb700 1 -- 172.21.15.71:0/3502513341 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(25 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7efc64013ec0 con 0x7efc680f5480 2024-02-20T12:53:26.119 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:26.108+0000 7efc5dffb700 1 -- 172.21.15.71:0/3502513341 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7efc64014d30 con 0x7efc680f5480 2024-02-20T12:53:26.119 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:26.108+0000 7efc6f727700 1 -- 172.21.15.71:0/3502513341 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc68106270 con 0x7efc680f5480 2024-02-20T12:53:26.119 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:26.109+0000 7efc5dffb700 1 -- 172.21.15.71:0/3502513341 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 37151+0+0 (secure 0 0 0) 0x7efc64014420 con 0x7efc680f5480 2024-02-20T12:53:26.120 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:26.109+0000 7efc5dffb700 1 --2- 172.21.15.71:0/3502513341 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7efc54031090 0x7efc54033550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:53:26.120 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:26.109+0000 7efc6f727700 1 -- 172.21.15.71:0/3502513341 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc68045bb0 con 0x7efc680f5480 2024-02-20T12:53:26.120 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:26.109+0000 7efc5dffb700 1 -- 172.21.15.71:0/3502513341 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1065+0+0 (secure 0 0 0) 0x7efc64042400 con 0x7efc680f5480 2024-02-20T12:53:26.120 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:26.109+0000 7efc6ccc2700 1 --2- 172.21.15.71:0/3502513341 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7efc54031090 0x7efc54033550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:53:26.120 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:26.110+0000 7efc6ccc2700 1 --2- 172.21.15.71:0/3502513341 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7efc54031090 0x7efc54033550 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7efc580096a0 tx=0x7efc58006b40).ready entity=mgr.14156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:53:26.120 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:26.114+0000 7efc5dffb700 1 -- 172.21.15.71:0/3502513341 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+153237 (secure 0 0 0) 0x7efc6401d980 con 0x7efc680f5480 2024-02-20T12:53:26.324 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:26.323+0000 7efc6f727700 1 -- 172.21.15.71:0/3502513341 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7efc680fb060 con 0x7efc680f5480 2024-02-20T12:53:26.324 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:26.324+0000 7efc5dffb700 1 -- 172.21.15.71:0/3502513341 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7efc640170c0 con 0x7efc680f5480 2024-02-20T12:53:26.325 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-20T12:53:26.325 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":1,"fsid":"680ee126-cfee-11ee-95bb-87774f69a715","modified":"2024-02-20T12:49:44.699059Z","created":"2024-02-20T12:49:44.699059Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi064","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:3300","nonce":0},{"type":"v1","addr":"172.21.15.64:6789","nonce":0}]},"addr":"172.21.15.64:6789/0","public_addr":"172.21.15.64:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-20T12:53:26.326 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:26.326+0000 7efc6f727700 1 -- 172.21.15.71:0/3502513341 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7efc54031090 msgr2=0x7efc54033550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:53:26.326 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:26.326+0000 7efc6f727700 1 --2- 172.21.15.71:0/3502513341 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7efc54031090 0x7efc54033550 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7efc580096a0 tx=0x7efc58006b40).stop 2024-02-20T12:53:26.326 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:26.326+0000 7efc6f727700 1 -- 172.21.15.71:0/3502513341 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efc680f5480 msgr2=0x7efc681056f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:53:26.326 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:26.326+0000 7efc6f727700 1 --2- 172.21.15.71:0/3502513341 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efc680f5480 0x7efc681056f0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 rx=0x7efc640089f0 tx=0x7efc6400b040).stop 2024-02-20T12:53:26.327 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:26.326+0000 7efc6f727700 1 -- 172.21.15.71:0/3502513341 shutdown_connections 2024-02-20T12:53:26.327 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:26.326+0000 7efc6f727700 1 --2- 172.21.15.71:0/3502513341 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7efc54031090 0x7efc54033550 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:53:26.327 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:26.326+0000 7efc6f727700 1 --2- 172.21.15.71:0/3502513341 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efc680f5480 0x7efc681056f0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:53:26.327 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:26.326+0000 7efc6f727700 1 -- 172.21.15.71:0/3502513341 >> 172.21.15.71:0/3502513341 conn(0x7efc680f0be0 msgr2=0x7efc680f16d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:53:26.327 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:26.326+0000 7efc6f727700 1 -- 172.21.15.71:0/3502513341 shutdown_connections 2024-02-20T12:53:26.327 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:26.326+0000 7efc6f727700 1 -- 172.21.15.71:0/3502513341 wait complete. 2024-02-20T12:53:26.327 INFO:teuthology.orchestra.run.smithi071.stderr:dumped monmap epoch 1 2024-02-20T12:53:27.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:53:27 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:53:25.726225+0000 mgr.smithi064.vriyor (mgr.14156) 78 : cluster [DBG] pgmap v70: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:53:27.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:53:27 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:53:26.324714+0000 mon.smithi064 (mon.0) 142 : audit [DBG] from='client.? 172.21.15.71:0/3502513341' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-20T12:53:27.904 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-20T12:53:27.904 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/daemon-base:latest-pacific shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 680ee126-cfee-11ee-95bb-87774f69a715 -- ceph mon dump -f json 2024-02-20T12:53:29.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:53:29 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:53:27.726499+0000 mgr.smithi064.vriyor (mgr.14156) 79 : cluster [DBG] pgmap v71: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:53:29.759 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:29.740+0000 7f2d7d758700 1 -- 172.21.15.71:0/992744304 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2d780f6aa0 msgr2=0x7f2d780f6ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:53:29.759 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:29.740+0000 7f2d7d758700 1 --2- 172.21.15.71:0/992744304 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2d780f6aa0 0x7f2d780f6ec0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 rx=0x7f2d6c009fb0 tx=0x7f2d6c009b30).stop 2024-02-20T12:53:29.759 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:29.741+0000 7f2d7d758700 1 -- 172.21.15.71:0/992744304 shutdown_connections 2024-02-20T12:53:29.759 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:29.741+0000 7f2d7d758700 1 --2- 172.21.15.71:0/992744304 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2d780f6aa0 0x7f2d780f6ec0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:53:29.760 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:29.741+0000 7f2d7d758700 1 -- 172.21.15.71:0/992744304 >> 172.21.15.71:0/992744304 conn(0x7f2d780f0390 msgr2=0x7f2d780f27b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:53:29.760 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:29.741+0000 7f2d7d758700 1 -- 172.21.15.71:0/992744304 shutdown_connections 2024-02-20T12:53:29.760 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:29.741+0000 7f2d7d758700 1 -- 172.21.15.71:0/992744304 wait complete. 2024-02-20T12:53:29.760 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:29.741+0000 7f2d7d758700 1 Processor -- start 2024-02-20T12:53:29.760 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:29.741+0000 7f2d7d758700 1 -- start start 2024-02-20T12:53:29.760 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:29.741+0000 7f2d7d758700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2d780f6aa0 0x7f2d780fd490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:53:29.760 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:29.741+0000 7f2d7d758700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d780fb910 con 0x7f2d780f6aa0 2024-02-20T12:53:29.760 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:29.742+0000 7f2d76ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2d780f6aa0 0x7f2d780fd490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:53:29.760 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:29.742+0000 7f2d76ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2d780f6aa0 0x7f2d780fd490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.71:40982/0 (socket says 172.21.15.71:40982) 2024-02-20T12:53:29.760 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:29.742+0000 7f2d76ffd700 1 -- 172.21.15.71:0/220609933 learned_addr learned my addr 172.21.15.71:0/220609933 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-02-20T12:53:29.760 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:29.742+0000 7f2d76ffd700 1 -- 172.21.15.71:0/220609933 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d6c005040 con 0x7f2d780f6aa0 2024-02-20T12:53:29.760 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:29.742+0000 7f2d76ffd700 1 --2- 172.21.15.71:0/220609933 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2d780f6aa0 0x7f2d780fd490 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 rx=0x7f2d6c0089f0 tx=0x7f2d6c00b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:53:29.760 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:29.742+0000 7f2d5f7fe700 1 -- 172.21.15.71:0/220609933 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2d6c009320 con 0x7f2d780f6aa0 2024-02-20T12:53:29.761 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:29.742+0000 7f2d7d758700 1 -- 172.21.15.71:0/220609933 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d780fbab0 con 0x7f2d780f6aa0 2024-02-20T12:53:29.761 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:29.742+0000 7f2d5f7fe700 1 -- 172.21.15.71:0/220609933 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(25 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f2d6c013ec0 con 0x7f2d780f6aa0 2024-02-20T12:53:29.761 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:29.742+0000 7f2d5f7fe700 1 -- 172.21.15.71:0/220609933 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2d6c014d30 con 0x7f2d780f6aa0 2024-02-20T12:53:29.761 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:29.742+0000 7f2d7d758700 1 -- 172.21.15.71:0/220609933 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d780fbf40 con 0x7f2d780f6aa0 2024-02-20T12:53:29.761 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:29.743+0000 7f2d5f7fe700 1 -- 172.21.15.71:0/220609933 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 37151+0+0 (secure 0 0 0) 0x7f2d6c0095f0 con 0x7f2d780f6aa0 2024-02-20T12:53:29.761 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:29.743+0000 7f2d5f7fe700 1 --2- 172.21.15.71:0/220609933 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f2d600310e0 0x7f2d600335a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:53:29.761 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:29.743+0000 7f2d7d758700 1 -- 172.21.15.71:0/220609933 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d780591e0 con 0x7f2d780f6aa0 2024-02-20T12:53:29.761 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:29.743+0000 7f2d5f7fe700 1 -- 172.21.15.71:0/220609933 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1065+0+0 (secure 0 0 0) 0x7f2d6c042500 con 0x7f2d780f6aa0 2024-02-20T12:53:29.761 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:29.743+0000 7f2d767fc700 1 --2- 172.21.15.71:0/220609933 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f2d600310e0 0x7f2d600335a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:53:29.761 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:29.744+0000 7f2d767fc700 1 --2- 172.21.15.71:0/220609933 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f2d600310e0 0x7f2d600335a0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7f2d680096a0 tx=0x7f2d68006b40).ready entity=mgr.14156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:53:29.761 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:29.748+0000 7f2d5f7fe700 1 -- 172.21.15.71:0/220609933 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+153237 (secure 0 0 0) 0x7f2d6c01d7f0 con 0x7f2d780f6aa0 2024-02-20T12:53:29.956 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:29.955+0000 7f2d7d758700 1 -- 172.21.15.71:0/220609933 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f2d78045bb0 con 0x7f2d780f6aa0 2024-02-20T12:53:29.956 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:29.956+0000 7f2d5f7fe700 1 -- 172.21.15.71:0/220609933 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7f2d6c0170c0 con 0x7f2d780f6aa0 2024-02-20T12:53:29.957 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-20T12:53:29.957 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":1,"fsid":"680ee126-cfee-11ee-95bb-87774f69a715","modified":"2024-02-20T12:49:44.699059Z","created":"2024-02-20T12:49:44.699059Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi064","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:3300","nonce":0},{"type":"v1","addr":"172.21.15.64:6789","nonce":0}]},"addr":"172.21.15.64:6789/0","public_addr":"172.21.15.64:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-20T12:53:29.958 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:29.957+0000 7f2d7d758700 1 -- 172.21.15.71:0/220609933 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f2d600310e0 msgr2=0x7f2d600335a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:53:29.958 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:29.957+0000 7f2d7d758700 1 --2- 172.21.15.71:0/220609933 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f2d600310e0 0x7f2d600335a0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7f2d680096a0 tx=0x7f2d68006b40).stop 2024-02-20T12:53:29.958 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:29.957+0000 7f2d7d758700 1 -- 172.21.15.71:0/220609933 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2d780f6aa0 msgr2=0x7f2d780fd490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:53:29.958 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:29.957+0000 7f2d7d758700 1 --2- 172.21.15.71:0/220609933 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2d780f6aa0 0x7f2d780fd490 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 rx=0x7f2d6c0089f0 tx=0x7f2d6c00b040).stop 2024-02-20T12:53:29.958 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:29.957+0000 7f2d7d758700 1 -- 172.21.15.71:0/220609933 shutdown_connections 2024-02-20T12:53:29.958 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:29.957+0000 7f2d7d758700 1 --2- 172.21.15.71:0/220609933 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f2d600310e0 0x7f2d600335a0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:53:29.958 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:29.957+0000 7f2d7d758700 1 --2- 172.21.15.71:0/220609933 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2d780f6aa0 0x7f2d780fd490 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:53:29.959 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:29.957+0000 7f2d7d758700 1 -- 172.21.15.71:0/220609933 >> 172.21.15.71:0/220609933 conn(0x7f2d780f0390 msgr2=0x7f2d780f0e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:53:29.959 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:29.957+0000 7f2d7d758700 1 -- 172.21.15.71:0/220609933 shutdown_connections 2024-02-20T12:53:29.959 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:29.957+0000 7f2d7d758700 1 -- 172.21.15.71:0/220609933 wait complete. 2024-02-20T12:53:29.959 INFO:teuthology.orchestra.run.smithi071.stderr:dumped monmap epoch 1 2024-02-20T12:53:30.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:53:30 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:53:29.956646+0000 mon.smithi064 (mon.0) 143 : audit [DBG] from='client.? 172.21.15.71:0/220609933' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-20T12:53:31.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:53:31 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:53:29.726817+0000 mgr.smithi064.vriyor (mgr.14156) 80 : cluster [DBG] pgmap v72: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:53:31.695 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-20T12:53:31.696 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/daemon-base:latest-pacific shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 680ee126-cfee-11ee-95bb-87774f69a715 -- ceph mon dump -f json 2024-02-20T12:53:33.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:53:33 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:53:31.727090+0000 mgr.smithi064.vriyor (mgr.14156) 81 : cluster [DBG] pgmap v73: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:53:33.672 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:33.670+0000 7f6b1ff7a700 1 -- 172.21.15.71:0/2720688873 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6b180f5480 msgr2=0x7f6b180f58a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:53:33.673 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:33.670+0000 7f6b1ff7a700 1 --2- 172.21.15.71:0/2720688873 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6b180f5480 0x7f6b180f58a0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 rx=0x7f6b0c009fb0 tx=0x7f6b0c009b30).stop 2024-02-20T12:53:33.673 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:33.670+0000 7f6b1ff7a700 1 -- 172.21.15.71:0/2720688873 shutdown_connections 2024-02-20T12:53:33.673 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:33.670+0000 7f6b1ff7a700 1 --2- 172.21.15.71:0/2720688873 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6b180f5480 0x7f6b180f58a0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:53:33.673 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:33.670+0000 7f6b1ff7a700 1 -- 172.21.15.71:0/2720688873 >> 172.21.15.71:0/2720688873 conn(0x7f6b180f0be0 msgr2=0x7f6b180f3000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:53:33.673 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:33.670+0000 7f6b1ff7a700 1 -- 172.21.15.71:0/2720688873 shutdown_connections 2024-02-20T12:53:33.673 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:33.670+0000 7f6b1ff7a700 1 -- 172.21.15.71:0/2720688873 wait complete. 2024-02-20T12:53:33.673 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:33.671+0000 7f6b1ff7a700 1 Processor -- start 2024-02-20T12:53:33.673 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:33.671+0000 7f6b1ff7a700 1 -- start start 2024-02-20T12:53:33.673 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:33.671+0000 7f6b1ff7a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6b180f5480 0x7f6b181094f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:53:33.673 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:33.671+0000 7f6b1ff7a700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b18060320 con 0x7f6b180f5480 2024-02-20T12:53:33.673 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:33.671+0000 7f6b1dd16700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6b180f5480 0x7f6b181094f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:53:33.673 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:33.671+0000 7f6b1dd16700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6b180f5480 0x7f6b181094f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.71:41012/0 (socket says 172.21.15.71:41012) 2024-02-20T12:53:33.674 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:33.671+0000 7f6b1dd16700 1 -- 172.21.15.71:0/1974726353 learned_addr learned my addr 172.21.15.71:0/1974726353 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-02-20T12:53:33.674 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:33.672+0000 7f6b1dd16700 1 -- 172.21.15.71:0/1974726353 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b0c005040 con 0x7f6b180f5480 2024-02-20T12:53:33.674 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:33.672+0000 7f6b1dd16700 1 --2- 172.21.15.71:0/1974726353 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6b180f5480 0x7f6b181094f0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 rx=0x7f6b0c0089f0 tx=0x7f6b0c00b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:53:33.674 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:33.672+0000 7f6b0a7fc700 1 -- 172.21.15.71:0/1974726353 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6b0c009320 con 0x7f6b180f5480 2024-02-20T12:53:33.674 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:33.672+0000 7f6b1ff7a700 1 -- 172.21.15.71:0/1974726353 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b18060520 con 0x7f6b180f5480 2024-02-20T12:53:33.674 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:33.672+0000 7f6b0a7fc700 1 -- 172.21.15.71:0/1974726353 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(25 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f6b0c013ec0 con 0x7f6b180f5480 2024-02-20T12:53:33.674 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:33.672+0000 7f6b0a7fc700 1 -- 172.21.15.71:0/1974726353 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6b0c014d30 con 0x7f6b180f5480 2024-02-20T12:53:33.674 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:33.672+0000 7f6b1ff7a700 1 -- 172.21.15.71:0/1974726353 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b18109ba0 con 0x7f6b180f5480 2024-02-20T12:53:33.675 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:33.673+0000 7f6b0a7fc700 1 -- 172.21.15.71:0/1974726353 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 37151+0+0 (secure 0 0 0) 0x7f6b0c0095f0 con 0x7f6b180f5480 2024-02-20T12:53:33.675 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:33.673+0000 7f6b1ff7a700 1 -- 172.21.15.71:0/1974726353 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b18045bb0 con 0x7f6b180f5480 2024-02-20T12:53:33.675 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:33.673+0000 7f6b0a7fc700 1 --2- 172.21.15.71:0/1974726353 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f6b04031090 0x7f6b04033550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:53:33.675 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:33.673+0000 7f6b0a7fc700 1 -- 172.21.15.71:0/1974726353 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1065+0+0 (secure 0 0 0) 0x7f6b0c042420 con 0x7f6b180f5480 2024-02-20T12:53:33.675 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:33.673+0000 7f6b1d515700 1 --2- 172.21.15.71:0/1974726353 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f6b04031090 0x7f6b04033550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:53:33.675 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:33.674+0000 7f6b1d515700 1 --2- 172.21.15.71:0/1974726353 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f6b04031090 0x7f6b04033550 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0x7f6b140096a0 tx=0x7f6b14006b40).ready entity=mgr.14156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:53:33.678 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:33.678+0000 7f6b0a7fc700 1 -- 172.21.15.71:0/1974726353 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+153237 (secure 0 0 0) 0x7f6b0c01d7f0 con 0x7f6b180f5480 2024-02-20T12:53:33.886 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:33.886+0000 7f6b1ff7a700 1 -- 172.21.15.71:0/1974726353 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f6b180faf70 con 0x7f6b180f5480 2024-02-20T12:53:33.887 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:33.887+0000 7f6b0a7fc700 1 -- 172.21.15.71:0/1974726353 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7f6b0c00fe40 con 0x7f6b180f5480 2024-02-20T12:53:33.888 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-20T12:53:33.888 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":1,"fsid":"680ee126-cfee-11ee-95bb-87774f69a715","modified":"2024-02-20T12:49:44.699059Z","created":"2024-02-20T12:49:44.699059Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi064","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:3300","nonce":0},{"type":"v1","addr":"172.21.15.64:6789","nonce":0}]},"addr":"172.21.15.64:6789/0","public_addr":"172.21.15.64:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-20T12:53:33.889 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:33.888+0000 7f6b1ff7a700 1 -- 172.21.15.71:0/1974726353 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f6b04031090 msgr2=0x7f6b04033550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:53:33.889 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:33.888+0000 7f6b1ff7a700 1 --2- 172.21.15.71:0/1974726353 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f6b04031090 0x7f6b04033550 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0x7f6b140096a0 tx=0x7f6b14006b40).stop 2024-02-20T12:53:33.889 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:33.888+0000 7f6b1ff7a700 1 -- 172.21.15.71:0/1974726353 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6b180f5480 msgr2=0x7f6b181094f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:53:33.889 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:33.888+0000 7f6b1ff7a700 1 --2- 172.21.15.71:0/1974726353 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6b180f5480 0x7f6b181094f0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 rx=0x7f6b0c0089f0 tx=0x7f6b0c00b040).stop 2024-02-20T12:53:33.889 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:33.888+0000 7f6b1ff7a700 1 -- 172.21.15.71:0/1974726353 shutdown_connections 2024-02-20T12:53:33.889 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:33.888+0000 7f6b1ff7a700 1 --2- 172.21.15.71:0/1974726353 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f6b04031090 0x7f6b04033550 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:53:33.889 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:33.888+0000 7f6b1ff7a700 1 --2- 172.21.15.71:0/1974726353 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6b180f5480 0x7f6b181094f0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:53:33.889 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:33.888+0000 7f6b1ff7a700 1 -- 172.21.15.71:0/1974726353 >> 172.21.15.71:0/1974726353 conn(0x7f6b180f0be0 msgr2=0x7f6b180f16d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:53:33.889 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:33.888+0000 7f6b1ff7a700 1 -- 172.21.15.71:0/1974726353 shutdown_connections 2024-02-20T12:53:33.889 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:33.888+0000 7f6b1ff7a700 1 -- 172.21.15.71:0/1974726353 wait complete. 2024-02-20T12:53:33.890 INFO:teuthology.orchestra.run.smithi071.stderr:dumped monmap epoch 1 2024-02-20T12:53:34.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:53:34 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:53:33.887285+0000 mon.smithi064 (mon.0) 144 : audit [DBG] from='client.? 172.21.15.71:0/1974726353' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-20T12:53:35.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:53:35 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:53:33.727289+0000 mgr.smithi064.vriyor (mgr.14156) 82 : cluster [DBG] pgmap v74: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:53:35.556 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-20T12:53:35.556 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/daemon-base:latest-pacific shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 680ee126-cfee-11ee-95bb-87774f69a715 -- ceph mon dump -f json 2024-02-20T12:53:37.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:53:37 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:53:35.727637+0000 mgr.smithi064.vriyor (mgr.14156) 83 : cluster [DBG] pgmap v75: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:53:37.463 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:37.454+0000 7f952f4ea700 1 -- 172.21.15.71:0/2034592427 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9528060360 msgr2=0x7f95280f9160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:53:37.463 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:37.454+0000 7f952f4ea700 1 --2- 172.21.15.71:0/2034592427 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9528060360 0x7f95280f9160 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 rx=0x7f9518009fb0 tx=0x7f9518009b30).stop 2024-02-20T12:53:37.464 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:37.454+0000 7f952f4ea700 1 -- 172.21.15.71:0/2034592427 shutdown_connections 2024-02-20T12:53:37.464 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:37.454+0000 7f952f4ea700 1 --2- 172.21.15.71:0/2034592427 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9528060360 0x7f95280f9160 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:53:37.464 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:37.454+0000 7f952f4ea700 1 -- 172.21.15.71:0/2034592427 >> 172.21.15.71:0/2034592427 conn(0x7f95280f0be0 msgr2=0x7f95280f3000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:53:37.464 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:37.454+0000 7f952f4ea700 1 -- 172.21.15.71:0/2034592427 shutdown_connections 2024-02-20T12:53:37.464 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:37.454+0000 7f952f4ea700 1 -- 172.21.15.71:0/2034592427 wait complete. 2024-02-20T12:53:37.464 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:37.455+0000 7f952f4ea700 1 Processor -- start 2024-02-20T12:53:37.465 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:37.455+0000 7f952f4ea700 1 -- start start 2024-02-20T12:53:37.465 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:37.455+0000 7f952f4ea700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9528060360 0x7f9528106750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:53:37.465 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:37.455+0000 7f952f4ea700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9528106c40 con 0x7f9528060360 2024-02-20T12:53:37.465 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:37.455+0000 7f952d286700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9528060360 0x7f9528106750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:53:37.465 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:37.455+0000 7f952d286700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9528060360 0x7f9528106750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.71:47520/0 (socket says 172.21.15.71:47520) 2024-02-20T12:53:37.465 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:37.455+0000 7f952d286700 1 -- 172.21.15.71:0/533466915 learned_addr learned my addr 172.21.15.71:0/533466915 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-02-20T12:53:37.465 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:37.455+0000 7f952d286700 1 -- 172.21.15.71:0/533466915 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9518005040 con 0x7f9528060360 2024-02-20T12:53:37.466 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:37.455+0000 7f952d286700 1 --2- 172.21.15.71:0/533466915 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9528060360 0x7f9528106750 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 rx=0x7f9518009f80 tx=0x7f951800b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:53:37.466 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:37.455+0000 7f951dffb700 1 -- 172.21.15.71:0/533466915 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9518009320 con 0x7f9528060360 2024-02-20T12:53:37.466 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:37.456+0000 7f952f4ea700 1 -- 172.21.15.71:0/533466915 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9528106e40 con 0x7f9528060360 2024-02-20T12:53:37.466 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:37.456+0000 7f951dffb700 1 -- 172.21.15.71:0/533466915 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(25 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f9518013680 con 0x7f9528060360 2024-02-20T12:53:37.466 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:37.456+0000 7f951dffb700 1 -- 172.21.15.71:0/533466915 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f951801eda0 con 0x7f9528060360 2024-02-20T12:53:37.466 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:37.456+0000 7f952f4ea700 1 -- 172.21.15.71:0/533466915 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95281072d0 con 0x7f9528060360 2024-02-20T12:53:37.466 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:37.457+0000 7f951dffb700 1 -- 172.21.15.71:0/533466915 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 37151+0+0 (secure 0 0 0) 0x7f951801e420 con 0x7f9528060360 2024-02-20T12:53:37.467 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:37.457+0000 7f952f4ea700 1 -- 172.21.15.71:0/533466915 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9528045bb0 con 0x7f9528060360 2024-02-20T12:53:37.467 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:37.457+0000 7f951dffb700 1 --2- 172.21.15.71:0/533466915 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f9514031090 0x7f9514033550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:53:37.467 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:37.457+0000 7f951dffb700 1 -- 172.21.15.71:0/533466915 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1065+0+0 (secure 0 0 0) 0x7f9518042420 con 0x7f9528060360 2024-02-20T12:53:37.467 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:37.457+0000 7f952ca85700 1 --2- 172.21.15.71:0/533466915 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f9514031090 0x7f9514033550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:53:37.467 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:37.457+0000 7f952ca85700 1 --2- 172.21.15.71:0/533466915 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f9514031090 0x7f9514033550 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7f95240096a0 tx=0x7f9524006b40).ready entity=mgr.14156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:53:37.467 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:37.461+0000 7f951dffb700 1 -- 172.21.15.71:0/533466915 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+153237 (secure 0 0 0) 0x7f951801d380 con 0x7f9528060360 2024-02-20T12:53:37.669 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:37.669+0000 7f952f4ea700 1 -- 172.21.15.71:0/533466915 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f9528108180 con 0x7f9528060360 2024-02-20T12:53:37.670 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:37.669+0000 7f951dffb700 1 -- 172.21.15.71:0/533466915 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7f9518004490 con 0x7f9528060360 2024-02-20T12:53:37.670 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-20T12:53:37.670 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":1,"fsid":"680ee126-cfee-11ee-95bb-87774f69a715","modified":"2024-02-20T12:49:44.699059Z","created":"2024-02-20T12:49:44.699059Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi064","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:3300","nonce":0},{"type":"v1","addr":"172.21.15.64:6789","nonce":0}]},"addr":"172.21.15.64:6789/0","public_addr":"172.21.15.64:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-20T12:53:37.671 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:37.671+0000 7f952f4ea700 1 -- 172.21.15.71:0/533466915 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f9514031090 msgr2=0x7f9514033550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:53:37.671 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:37.671+0000 7f952f4ea700 1 --2- 172.21.15.71:0/533466915 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f9514031090 0x7f9514033550 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7f95240096a0 tx=0x7f9524006b40).stop 2024-02-20T12:53:37.672 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:37.671+0000 7f952f4ea700 1 -- 172.21.15.71:0/533466915 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9528060360 msgr2=0x7f9528106750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:53:37.672 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:37.671+0000 7f952f4ea700 1 --2- 172.21.15.71:0/533466915 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9528060360 0x7f9528106750 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 rx=0x7f9518009f80 tx=0x7f951800b040).stop 2024-02-20T12:53:37.672 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:37.671+0000 7f952f4ea700 1 -- 172.21.15.71:0/533466915 shutdown_connections 2024-02-20T12:53:37.672 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:37.671+0000 7f952f4ea700 1 --2- 172.21.15.71:0/533466915 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f9514031090 0x7f9514033550 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:53:37.672 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:37.671+0000 7f952f4ea700 1 --2- 172.21.15.71:0/533466915 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9528060360 0x7f9528106750 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:53:37.672 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:37.671+0000 7f952f4ea700 1 -- 172.21.15.71:0/533466915 >> 172.21.15.71:0/533466915 conn(0x7f95280f0be0 msgr2=0x7f95280f1660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:53:37.672 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:37.671+0000 7f952f4ea700 1 -- 172.21.15.71:0/533466915 shutdown_connections 2024-02-20T12:53:37.672 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:37.671+0000 7f952f4ea700 1 -- 172.21.15.71:0/533466915 wait complete. 2024-02-20T12:53:37.672 INFO:teuthology.orchestra.run.smithi071.stderr:dumped monmap epoch 1 2024-02-20T12:53:38.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:53:38 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:53:37.670176+0000 mon.smithi064 (mon.0) 145 : audit [DBG] from='client.? 172.21.15.71:0/533466915' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-20T12:53:39.292 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-20T12:53:39.292 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/daemon-base:latest-pacific shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 680ee126-cfee-11ee-95bb-87774f69a715 -- ceph mon dump -f json 2024-02-20T12:53:39.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:53:39 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:53:37.727931+0000 mgr.smithi064.vriyor (mgr.14156) 84 : cluster [DBG] pgmap v76: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:53:41.191 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:41.189+0000 7f127ee3d700 1 -- 172.21.15.71:0/3068303005 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f12780f6590 msgr2=0x7f12780f69b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:53:41.191 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:41.189+0000 7f127ee3d700 1 --2- 172.21.15.71:0/3068303005 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f12780f6590 0x7f12780f69b0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 rx=0x7f126c004760 tx=0x7f126c009ad0).stop 2024-02-20T12:53:41.191 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:41.189+0000 7f127ee3d700 1 -- 172.21.15.71:0/3068303005 shutdown_connections 2024-02-20T12:53:41.191 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:41.189+0000 7f127ee3d700 1 --2- 172.21.15.71:0/3068303005 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f12780f6590 0x7f12780f69b0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:53:41.191 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:41.189+0000 7f127ee3d700 1 -- 172.21.15.71:0/3068303005 >> 172.21.15.71:0/3068303005 conn(0x7f12780f1cd0 msgr2=0x7f12780f4110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:53:41.191 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:41.189+0000 7f127ee3d700 1 -- 172.21.15.71:0/3068303005 shutdown_connections 2024-02-20T12:53:41.192 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:41.189+0000 7f127ee3d700 1 -- 172.21.15.71:0/3068303005 wait complete. 2024-02-20T12:53:41.192 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:41.190+0000 7f127ee3d700 1 Processor -- start 2024-02-20T12:53:41.192 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:41.190+0000 7f127ee3d700 1 -- start start 2024-02-20T12:53:41.192 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:41.190+0000 7f127ee3d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f12780f6590 0x7f12781067c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:53:41.192 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:41.190+0000 7f127ee3d700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1278106cb0 con 0x7f12780f6590 2024-02-20T12:53:41.192 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:41.190+0000 7f127cbd9700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f12780f6590 0x7f12781067c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:53:41.192 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:41.190+0000 7f127cbd9700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f12780f6590 0x7f12781067c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.71:47548/0 (socket says 172.21.15.71:47548) 2024-02-20T12:53:41.192 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:41.190+0000 7f127cbd9700 1 -- 172.21.15.71:0/898072695 learned_addr learned my addr 172.21.15.71:0/898072695 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-02-20T12:53:41.192 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:41.191+0000 7f127cbd9700 1 -- 172.21.15.71:0/898072695 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f126c009d70 con 0x7f12780f6590 2024-02-20T12:53:41.192 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:41.191+0000 7f127cbd9700 1 --2- 172.21.15.71:0/898072695 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f12780f6590 0x7f12781067c0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 rx=0x7f126c004170 tx=0x7f126c009ad0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:53:41.193 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:41.191+0000 7f12757fa700 1 -- 172.21.15.71:0/898072695 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f126c00e040 con 0x7f12780f6590 2024-02-20T12:53:41.194 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:41.191+0000 7f12757fa700 1 -- 172.21.15.71:0/898072695 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(25 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f126c008ef0 con 0x7f12780f6590 2024-02-20T12:53:41.194 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:41.191+0000 7f127ee3d700 1 -- 172.21.15.71:0/898072695 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1278106eb0 con 0x7f12780f6590 2024-02-20T12:53:41.194 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:41.191+0000 7f12757fa700 1 -- 172.21.15.71:0/898072695 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f126c014c90 con 0x7f12780f6590 2024-02-20T12:53:41.194 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:41.191+0000 7f127ee3d700 1 -- 172.21.15.71:0/898072695 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1278107340 con 0x7f12780f6590 2024-02-20T12:53:41.194 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:41.192+0000 7f12757fa700 1 -- 172.21.15.71:0/898072695 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 37151+0+0 (secure 0 0 0) 0x7f126c014420 con 0x7f12780f6590 2024-02-20T12:53:41.194 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:41.192+0000 7f12757fa700 1 --2- 172.21.15.71:0/898072695 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f1268031090 0x7f1268033550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:53:41.194 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:41.192+0000 7f12757fa700 1 -- 172.21.15.71:0/898072695 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1065+0+0 (secure 0 0 0) 0x7f126c0421f0 con 0x7f12780f6590 2024-02-20T12:53:41.194 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:41.192+0000 7f127ee3d700 1 -- 172.21.15.71:0/898072695 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1278045bb0 con 0x7f12780f6590 2024-02-20T12:53:41.194 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:41.192+0000 7f1277fff700 1 --2- 172.21.15.71:0/898072695 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f1268031090 0x7f1268033550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:53:41.194 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:41.193+0000 7f1277fff700 1 --2- 172.21.15.71:0/898072695 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f1268031090 0x7f1268033550 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7f12640096a0 tx=0x7f1264006b40).ready entity=mgr.14156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:53:41.197 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:41.197+0000 7f12757fa700 1 -- 172.21.15.71:0/898072695 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+153237 (secure 0 0 0) 0x7f126c026300 con 0x7f12780f6590 2024-02-20T12:53:41.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:53:41 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:53:39.728264+0000 mgr.smithi064.vriyor (mgr.14156) 85 : cluster [DBG] pgmap v77: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:53:41.402 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:41.401+0000 7f127ee3d700 1 -- 172.21.15.71:0/898072695 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f1278108060 con 0x7f12780f6590 2024-02-20T12:53:41.404 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:41.402+0000 7f12757fa700 1 -- 172.21.15.71:0/898072695 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7f126c012b20 con 0x7f12780f6590 2024-02-20T12:53:41.404 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-20T12:53:41.404 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":1,"fsid":"680ee126-cfee-11ee-95bb-87774f69a715","modified":"2024-02-20T12:49:44.699059Z","created":"2024-02-20T12:49:44.699059Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi064","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:3300","nonce":0},{"type":"v1","addr":"172.21.15.64:6789","nonce":0}]},"addr":"172.21.15.64:6789/0","public_addr":"172.21.15.64:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-20T12:53:41.405 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:41.403+0000 7f127ee3d700 1 -- 172.21.15.71:0/898072695 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f1268031090 msgr2=0x7f1268033550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:53:41.405 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:41.403+0000 7f127ee3d700 1 --2- 172.21.15.71:0/898072695 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f1268031090 0x7f1268033550 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7f12640096a0 tx=0x7f1264006b40).stop 2024-02-20T12:53:41.405 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:41.403+0000 7f127ee3d700 1 -- 172.21.15.71:0/898072695 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f12780f6590 msgr2=0x7f12781067c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:53:41.405 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:41.403+0000 7f127ee3d700 1 --2- 172.21.15.71:0/898072695 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f12780f6590 0x7f12781067c0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 rx=0x7f126c004170 tx=0x7f126c009ad0).stop 2024-02-20T12:53:41.405 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:41.404+0000 7f127ee3d700 1 -- 172.21.15.71:0/898072695 shutdown_connections 2024-02-20T12:53:41.405 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:41.404+0000 7f127ee3d700 1 --2- 172.21.15.71:0/898072695 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f1268031090 0x7f1268033550 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:53:41.405 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:41.404+0000 7f127ee3d700 1 --2- 172.21.15.71:0/898072695 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f12780f6590 0x7f12781067c0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:53:41.406 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:41.404+0000 7f127ee3d700 1 -- 172.21.15.71:0/898072695 >> 172.21.15.71:0/898072695 conn(0x7f12780f1cd0 msgr2=0x7f12780f27c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:53:41.406 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:41.404+0000 7f127ee3d700 1 -- 172.21.15.71:0/898072695 shutdown_connections 2024-02-20T12:53:41.406 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:41.404+0000 7f127ee3d700 1 -- 172.21.15.71:0/898072695 wait complete. 2024-02-20T12:53:41.406 INFO:teuthology.orchestra.run.smithi071.stderr:dumped monmap epoch 1 2024-02-20T12:53:42.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:53:42 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:53:41.402683+0000 mon.smithi064 (mon.0) 146 : audit [DBG] from='client.? 172.21.15.71:0/898072695' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-20T12:53:43.066 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-20T12:53:43.067 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/daemon-base:latest-pacific shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 680ee126-cfee-11ee-95bb-87774f69a715 -- ceph mon dump -f json 2024-02-20T12:53:43.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:53:43 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:53:41.728625+0000 mgr.smithi064.vriyor (mgr.14156) 86 : cluster [DBG] pgmap v78: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:53:45.073 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:45.072+0000 7f33cd8e6700 1 -- 172.21.15.71:0/1507638304 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f33c80f54b0 msgr2=0x7f33c80f58d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:53:45.073 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:45.072+0000 7f33cd8e6700 1 --2- 172.21.15.71:0/1507638304 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f33c80f54b0 0x7f33c80f58d0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 rx=0x7f33b0009fb0 tx=0x7f33b0009b30).stop 2024-02-20T12:53:45.073 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:45.072+0000 7f33cd8e6700 1 -- 172.21.15.71:0/1507638304 shutdown_connections 2024-02-20T12:53:45.073 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:45.072+0000 7f33cd8e6700 1 --2- 172.21.15.71:0/1507638304 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f33c80f54b0 0x7f33c80f58d0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:53:45.073 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:45.072+0000 7f33cd8e6700 1 -- 172.21.15.71:0/1507638304 >> 172.21.15.71:0/1507638304 conn(0x7f33c80f0c10 msgr2=0x7f33c80f3030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:53:45.073 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:45.072+0000 7f33cd8e6700 1 -- 172.21.15.71:0/1507638304 shutdown_connections 2024-02-20T12:53:45.073 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:45.072+0000 7f33cd8e6700 1 -- 172.21.15.71:0/1507638304 wait complete. 2024-02-20T12:53:45.075 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:45.073+0000 7f33cd8e6700 1 Processor -- start 2024-02-20T12:53:45.075 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:45.073+0000 7f33cd8e6700 1 -- start start 2024-02-20T12:53:45.075 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:45.073+0000 7f33cd8e6700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f33c80f54b0 0x7f33c81094f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:53:45.076 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:45.073+0000 7f33cd8e6700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33c8060320 con 0x7f33c80f54b0 2024-02-20T12:53:45.076 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:45.073+0000 7f33c6ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f33c80f54b0 0x7f33c81094f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:53:45.076 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:45.073+0000 7f33c6ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f33c80f54b0 0x7f33c81094f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.71:47572/0 (socket says 172.21.15.71:47572) 2024-02-20T12:53:45.076 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:45.073+0000 7f33c6ffd700 1 -- 172.21.15.71:0/2487336243 learned_addr learned my addr 172.21.15.71:0/2487336243 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-02-20T12:53:45.076 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:45.073+0000 7f33c6ffd700 1 -- 172.21.15.71:0/2487336243 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33b0005040 con 0x7f33c80f54b0 2024-02-20T12:53:45.076 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:45.074+0000 7f33c6ffd700 1 --2- 172.21.15.71:0/2487336243 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f33c80f54b0 0x7f33c81094f0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 rx=0x7f33b0000c00 tx=0x7f33b000b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:53:45.076 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:45.074+0000 7f33bf7fe700 1 -- 172.21.15.71:0/2487336243 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f33b00095f0 con 0x7f33c80f54b0 2024-02-20T12:53:45.076 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:45.074+0000 7f33cd8e6700 1 -- 172.21.15.71:0/2487336243 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33c8060520 con 0x7f33c80f54b0 2024-02-20T12:53:45.076 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:45.074+0000 7f33bf7fe700 1 -- 172.21.15.71:0/2487336243 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(25 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f33b0007e50 con 0x7f33c80f54b0 2024-02-20T12:53:45.076 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:45.074+0000 7f33bf7fe700 1 -- 172.21.15.71:0/2487336243 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f33b0014e70 con 0x7f33c80f54b0 2024-02-20T12:53:45.076 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:45.074+0000 7f33cd8e6700 1 -- 172.21.15.71:0/2487336243 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33c8109ba0 con 0x7f33c80f54b0 2024-02-20T12:53:45.076 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:45.075+0000 7f33bf7fe700 1 -- 172.21.15.71:0/2487336243 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 37151+0+0 (secure 0 0 0) 0x7f33b0025070 con 0x7f33c80f54b0 2024-02-20T12:53:45.076 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:45.075+0000 7f33cd8e6700 1 -- 172.21.15.71:0/2487336243 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33c8045bb0 con 0x7f33c80f54b0 2024-02-20T12:53:45.077 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:45.075+0000 7f33bf7fe700 1 --2- 172.21.15.71:0/2487336243 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f33b4031090 0x7f33b4033550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:53:45.077 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:45.075+0000 7f33bf7fe700 1 -- 172.21.15.71:0/2487336243 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1065+0+0 (secure 0 0 0) 0x7f33b0041dc0 con 0x7f33c80f54b0 2024-02-20T12:53:45.077 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:45.075+0000 7f33c67fc700 1 --2- 172.21.15.71:0/2487336243 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f33b4031090 0x7f33b4033550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:53:45.077 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:45.075+0000 7f33c67fc700 1 --2- 172.21.15.71:0/2487336243 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f33b4031090 0x7f33b4033550 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7f33b80096a0 tx=0x7f33b8006b40).ready entity=mgr.14156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:53:45.080 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:45.080+0000 7f33bf7fe700 1 -- 172.21.15.71:0/2487336243 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+153237 (secure 0 0 0) 0x7f33b000f300 con 0x7f33c80f54b0 2024-02-20T12:53:45.288 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:45.287+0000 7f33cd8e6700 1 -- 172.21.15.71:0/2487336243 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f33c80fafa0 con 0x7f33c80f54b0 2024-02-20T12:53:45.289 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:45.288+0000 7f33bf7fe700 1 -- 172.21.15.71:0/2487336243 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7f33b000f300 con 0x7f33c80f54b0 2024-02-20T12:53:45.289 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-20T12:53:45.289 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":1,"fsid":"680ee126-cfee-11ee-95bb-87774f69a715","modified":"2024-02-20T12:49:44.699059Z","created":"2024-02-20T12:49:44.699059Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi064","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:3300","nonce":0},{"type":"v1","addr":"172.21.15.64:6789","nonce":0}]},"addr":"172.21.15.64:6789/0","public_addr":"172.21.15.64:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-20T12:53:45.291 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:45.290+0000 7f33cd8e6700 1 -- 172.21.15.71:0/2487336243 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f33b4031090 msgr2=0x7f33b4033550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:53:45.291 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:45.290+0000 7f33cd8e6700 1 --2- 172.21.15.71:0/2487336243 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f33b4031090 0x7f33b4033550 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7f33b80096a0 tx=0x7f33b8006b40).stop 2024-02-20T12:53:45.291 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:45.290+0000 7f33cd8e6700 1 -- 172.21.15.71:0/2487336243 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f33c80f54b0 msgr2=0x7f33c81094f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:53:45.291 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:45.290+0000 7f33cd8e6700 1 --2- 172.21.15.71:0/2487336243 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f33c80f54b0 0x7f33c81094f0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 rx=0x7f33b0000c00 tx=0x7f33b000b040).stop 2024-02-20T12:53:45.291 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:45.290+0000 7f33cd8e6700 1 -- 172.21.15.71:0/2487336243 shutdown_connections 2024-02-20T12:53:45.291 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:45.290+0000 7f33cd8e6700 1 --2- 172.21.15.71:0/2487336243 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f33b4031090 0x7f33b4033550 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:53:45.291 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:45.290+0000 7f33cd8e6700 1 --2- 172.21.15.71:0/2487336243 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f33c80f54b0 0x7f33c81094f0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:53:45.291 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:45.290+0000 7f33cd8e6700 1 -- 172.21.15.71:0/2487336243 >> 172.21.15.71:0/2487336243 conn(0x7f33c80f0c10 msgr2=0x7f33c80f16d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:53:45.291 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:45.290+0000 7f33cd8e6700 1 -- 172.21.15.71:0/2487336243 shutdown_connections 2024-02-20T12:53:45.291 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:45.290+0000 7f33cd8e6700 1 -- 172.21.15.71:0/2487336243 wait complete. 2024-02-20T12:53:45.292 INFO:teuthology.orchestra.run.smithi071.stderr:dumped monmap epoch 1 2024-02-20T12:53:45.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:53:45 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:53:43.729021+0000 mgr.smithi064.vriyor (mgr.14156) 87 : cluster [DBG] pgmap v79: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:53:46.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:53:46 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:53:45.288967+0000 mon.smithi064 (mon.0) 147 : audit [DBG] from='client.? 172.21.15.71:0/2487336243' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-20T12:53:46.973 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-20T12:53:46.973 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/daemon-base:latest-pacific shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 680ee126-cfee-11ee-95bb-87774f69a715 -- ceph mon dump -f json 2024-02-20T12:53:47.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:53:47 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:53:45.729412+0000 mgr.smithi064.vriyor (mgr.14156) 88 : cluster [DBG] pgmap v80: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:53:48.898 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:48.895+0000 7fc84bb1e700 1 -- 172.21.15.71:0/2962752827 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc8440f6590 msgr2=0x7fc8440f69b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:53:48.898 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:48.895+0000 7fc84bb1e700 1 --2- 172.21.15.71:0/2962752827 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc8440f6590 0x7fc8440f69b0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 rx=0x7fc834009fb0 tx=0x7fc834009b30).stop 2024-02-20T12:53:48.898 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:48.895+0000 7fc84bb1e700 1 -- 172.21.15.71:0/2962752827 shutdown_connections 2024-02-20T12:53:48.898 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:48.895+0000 7fc84bb1e700 1 --2- 172.21.15.71:0/2962752827 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc8440f6590 0x7fc8440f69b0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:53:48.898 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:48.895+0000 7fc84bb1e700 1 -- 172.21.15.71:0/2962752827 >> 172.21.15.71:0/2962752827 conn(0x7fc8440f1cf0 msgr2=0x7fc8440f4110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:53:48.898 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:48.895+0000 7fc84bb1e700 1 -- 172.21.15.71:0/2962752827 shutdown_connections 2024-02-20T12:53:48.898 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:48.895+0000 7fc84bb1e700 1 -- 172.21.15.71:0/2962752827 wait complete. 2024-02-20T12:53:48.898 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:48.896+0000 7fc84bb1e700 1 Processor -- start 2024-02-20T12:53:48.899 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:48.896+0000 7fc84bb1e700 1 -- start start 2024-02-20T12:53:48.899 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:48.896+0000 7fc84bb1e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc8440f6590 0x7fc8441067d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:53:48.899 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:48.896+0000 7fc84bb1e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc844106cc0 con 0x7fc8440f6590 2024-02-20T12:53:48.899 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:48.896+0000 7fc8498ba700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc8440f6590 0x7fc8441067d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:53:48.899 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:48.896+0000 7fc8498ba700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc8440f6590 0x7fc8441067d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.71:40676/0 (socket says 172.21.15.71:40676) 2024-02-20T12:53:48.899 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:48.896+0000 7fc8498ba700 1 -- 172.21.15.71:0/4119667982 learned_addr learned my addr 172.21.15.71:0/4119667982 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-02-20T12:53:48.899 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:48.896+0000 7fc8498ba700 1 -- 172.21.15.71:0/4119667982 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc834005040 con 0x7fc8440f6590 2024-02-20T12:53:48.899 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:48.896+0000 7fc8498ba700 1 --2- 172.21.15.71:0/4119667982 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc8440f6590 0x7fc8441067d0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 rx=0x7fc8340089f0 tx=0x7fc83400b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:53:48.899 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:48.896+0000 7fc83a7fc700 1 -- 172.21.15.71:0/4119667982 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc834009320 con 0x7fc8440f6590 2024-02-20T12:53:48.899 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:48.896+0000 7fc84bb1e700 1 -- 172.21.15.71:0/4119667982 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc844106ec0 con 0x7fc8440f6590 2024-02-20T12:53:48.899 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:48.896+0000 7fc83a7fc700 1 -- 172.21.15.71:0/4119667982 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(25 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7fc834013ec0 con 0x7fc8440f6590 2024-02-20T12:53:48.899 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:48.896+0000 7fc83a7fc700 1 -- 172.21.15.71:0/4119667982 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc834014d30 con 0x7fc8440f6590 2024-02-20T12:53:48.899 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:48.897+0000 7fc84bb1e700 1 -- 172.21.15.71:0/4119667982 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc844107350 con 0x7fc8440f6590 2024-02-20T12:53:48.900 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:48.897+0000 7fc83a7fc700 1 -- 172.21.15.71:0/4119667982 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 37151+0+0 (secure 0 0 0) 0x7fc8340095f0 con 0x7fc8440f6590 2024-02-20T12:53:48.900 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:48.897+0000 7fc84bb1e700 1 -- 172.21.15.71:0/4119667982 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc844045bb0 con 0x7fc8440f6590 2024-02-20T12:53:48.900 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:48.898+0000 7fc83a7fc700 1 --2- 172.21.15.71:0/4119667982 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fc830031090 0x7fc830033550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:53:48.901 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:48.898+0000 7fc83a7fc700 1 -- 172.21.15.71:0/4119667982 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1065+0+0 (secure 0 0 0) 0x7fc8340423e0 con 0x7fc8440f6590 2024-02-20T12:53:48.901 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:48.898+0000 7fc8490b9700 1 --2- 172.21.15.71:0/4119667982 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fc830031090 0x7fc830033550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:53:48.901 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:48.898+0000 7fc8490b9700 1 --2- 172.21.15.71:0/4119667982 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fc830031090 0x7fc830033550 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7fc8400096a0 tx=0x7fc840006b40).ready entity=mgr.14156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:53:48.904 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:48.902+0000 7fc83a7fc700 1 -- 172.21.15.71:0/4119667982 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+153237 (secure 0 0 0) 0x7fc83401d7f0 con 0x7fc8440f6590 2024-02-20T12:53:49.110 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:49.109+0000 7fc84bb1e700 1 -- 172.21.15.71:0/4119667982 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fc844108200 con 0x7fc8440f6590 2024-02-20T12:53:49.111 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:49.110+0000 7fc83a7fc700 1 -- 172.21.15.71:0/4119667982 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7fc83401d7f0 con 0x7fc8440f6590 2024-02-20T12:53:49.111 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-20T12:53:49.111 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":1,"fsid":"680ee126-cfee-11ee-95bb-87774f69a715","modified":"2024-02-20T12:49:44.699059Z","created":"2024-02-20T12:49:44.699059Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi064","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:3300","nonce":0},{"type":"v1","addr":"172.21.15.64:6789","nonce":0}]},"addr":"172.21.15.64:6789/0","public_addr":"172.21.15.64:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-20T12:53:49.113 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:49.111+0000 7fc84bb1e700 1 -- 172.21.15.71:0/4119667982 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fc830031090 msgr2=0x7fc830033550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:53:49.113 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:49.111+0000 7fc84bb1e700 1 --2- 172.21.15.71:0/4119667982 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fc830031090 0x7fc830033550 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7fc8400096a0 tx=0x7fc840006b40).stop 2024-02-20T12:53:49.113 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:49.111+0000 7fc84bb1e700 1 -- 172.21.15.71:0/4119667982 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc8440f6590 msgr2=0x7fc8441067d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:53:49.113 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:49.111+0000 7fc84bb1e700 1 --2- 172.21.15.71:0/4119667982 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc8440f6590 0x7fc8441067d0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 rx=0x7fc8340089f0 tx=0x7fc83400b040).stop 2024-02-20T12:53:49.113 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:49.111+0000 7fc84bb1e700 1 -- 172.21.15.71:0/4119667982 shutdown_connections 2024-02-20T12:53:49.113 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:49.111+0000 7fc84bb1e700 1 --2- 172.21.15.71:0/4119667982 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fc830031090 0x7fc830033550 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:53:49.113 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:49.111+0000 7fc84bb1e700 1 --2- 172.21.15.71:0/4119667982 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc8440f6590 0x7fc8441067d0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:53:49.113 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:49.111+0000 7fc84bb1e700 1 -- 172.21.15.71:0/4119667982 >> 172.21.15.71:0/4119667982 conn(0x7fc8440f1cf0 msgr2=0x7fc8440f2770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:53:49.113 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:49.111+0000 7fc84bb1e700 1 -- 172.21.15.71:0/4119667982 shutdown_connections 2024-02-20T12:53:49.114 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:49.111+0000 7fc84bb1e700 1 -- 172.21.15.71:0/4119667982 wait complete. 2024-02-20T12:53:49.114 INFO:teuthology.orchestra.run.smithi071.stderr:dumped monmap epoch 1 2024-02-20T12:53:49.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:53:49 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:53:47.729774+0000 mgr.smithi064.vriyor (mgr.14156) 89 : cluster [DBG] pgmap v81: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:53:50.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:53:50 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:53:49.111339+0000 mon.smithi064 (mon.0) 148 : audit [DBG] from='client.? 172.21.15.71:0/4119667982' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-20T12:53:50.791 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-20T12:53:50.792 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/daemon-base:latest-pacific shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 680ee126-cfee-11ee-95bb-87774f69a715 -- ceph mon dump -f json 2024-02-20T12:53:51.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:53:51 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:53:49.730165+0000 mgr.smithi064.vriyor (mgr.14156) 90 : cluster [DBG] pgmap v82: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:53:52.806 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:52.804+0000 7fbd7a65b700 1 -- 172.21.15.71:0/3357686789 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbd740f6580 msgr2=0x7fbd740f69a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:53:52.806 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:52.804+0000 7fbd7a65b700 1 --2- 172.21.15.71:0/3357686789 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbd740f6580 0x7fbd740f69a0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 rx=0x7fbd64009fb0 tx=0x7fbd64009b30).stop 2024-02-20T12:53:52.806 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:52.805+0000 7fbd7a65b700 1 -- 172.21.15.71:0/3357686789 shutdown_connections 2024-02-20T12:53:52.806 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:52.805+0000 7fbd7a65b700 1 --2- 172.21.15.71:0/3357686789 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbd740f6580 0x7fbd740f69a0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:53:52.807 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:52.805+0000 7fbd7a65b700 1 -- 172.21.15.71:0/3357686789 >> 172.21.15.71:0/3357686789 conn(0x7fbd740f1ce0 msgr2=0x7fbd740f4100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:53:52.807 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:52.805+0000 7fbd7a65b700 1 -- 172.21.15.71:0/3357686789 shutdown_connections 2024-02-20T12:53:52.807 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:52.805+0000 7fbd7a65b700 1 -- 172.21.15.71:0/3357686789 wait complete. 2024-02-20T12:53:52.807 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:52.805+0000 7fbd7a65b700 1 Processor -- start 2024-02-20T12:53:52.808 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:52.805+0000 7fbd7a65b700 1 -- start start 2024-02-20T12:53:52.808 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:52.805+0000 7fbd7a65b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbd740f6580 0x7fbd7406ba80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:53:52.808 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:52.805+0000 7fbd7a65b700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd74069f00 con 0x7fbd740f6580 2024-02-20T12:53:52.808 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:52.806+0000 7fbd73fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbd740f6580 0x7fbd7406ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:53:52.808 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:52.806+0000 7fbd73fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbd740f6580 0x7fbd7406ba80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.71:40702/0 (socket says 172.21.15.71:40702) 2024-02-20T12:53:52.808 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:52.806+0000 7fbd73fff700 1 -- 172.21.15.71:0/455536723 learned_addr learned my addr 172.21.15.71:0/455536723 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-02-20T12:53:52.808 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:52.806+0000 7fbd73fff700 1 -- 172.21.15.71:0/455536723 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd64005040 con 0x7fbd740f6580 2024-02-20T12:53:52.809 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:52.806+0000 7fbd73fff700 1 --2- 172.21.15.71:0/455536723 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbd740f6580 0x7fbd7406ba80 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 rx=0x7fbd640089f0 tx=0x7fbd6400b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:53:52.809 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:52.806+0000 7fbd70ff9700 1 -- 172.21.15.71:0/455536723 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fbd64009320 con 0x7fbd740f6580 2024-02-20T12:53:52.809 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:52.806+0000 7fbd7a65b700 1 -- 172.21.15.71:0/455536723 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd7406a0a0 con 0x7fbd740f6580 2024-02-20T12:53:52.809 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:52.806+0000 7fbd70ff9700 1 -- 172.21.15.71:0/455536723 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(25 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7fbd64013ec0 con 0x7fbd740f6580 2024-02-20T12:53:52.809 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:52.806+0000 7fbd70ff9700 1 -- 172.21.15.71:0/455536723 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fbd64014d30 con 0x7fbd740f6580 2024-02-20T12:53:52.809 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:52.806+0000 7fbd7a65b700 1 -- 172.21.15.71:0/455536723 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd7406a530 con 0x7fbd740f6580 2024-02-20T12:53:52.810 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:52.807+0000 7fbd70ff9700 1 -- 172.21.15.71:0/455536723 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 37151+0+0 (secure 0 0 0) 0x7fbd640095f0 con 0x7fbd740f6580 2024-02-20T12:53:52.810 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:52.807+0000 7fbd7a65b700 1 -- 172.21.15.71:0/455536723 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd740591e0 con 0x7fbd740f6580 2024-02-20T12:53:52.810 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:52.807+0000 7fbd70ff9700 1 --2- 172.21.15.71:0/455536723 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fbd60031090 0x7fbd60033550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:53:52.810 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:52.807+0000 7fbd70ff9700 1 -- 172.21.15.71:0/455536723 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1065+0+0 (secure 0 0 0) 0x7fbd640424a0 con 0x7fbd740f6580 2024-02-20T12:53:52.810 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:52.807+0000 7fbd737fe700 1 --2- 172.21.15.71:0/455536723 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fbd60031090 0x7fbd60033550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:53:52.810 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:52.808+0000 7fbd737fe700 1 --2- 172.21.15.71:0/455536723 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fbd60031090 0x7fbd60033550 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7fbd5c0096a0 tx=0x7fbd5c006b40).ready entity=mgr.14156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:53:52.813 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:52.812+0000 7fbd70ff9700 1 -- 172.21.15.71:0/455536723 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+153237 (secure 0 0 0) 0x7fbd6401d7f0 con 0x7fbd740f6580 2024-02-20T12:53:53.023 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:53.022+0000 7fbd7a65b700 1 -- 172.21.15.71:0/455536723 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fbd740fc070 con 0x7fbd740f6580 2024-02-20T12:53:53.024 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:53.022+0000 7fbd70ff9700 1 -- 172.21.15.71:0/455536723 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7fbd6401d7f0 con 0x7fbd740f6580 2024-02-20T12:53:53.024 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-20T12:53:53.024 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":1,"fsid":"680ee126-cfee-11ee-95bb-87774f69a715","modified":"2024-02-20T12:49:44.699059Z","created":"2024-02-20T12:49:44.699059Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi064","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:3300","nonce":0},{"type":"v1","addr":"172.21.15.64:6789","nonce":0}]},"addr":"172.21.15.64:6789/0","public_addr":"172.21.15.64:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-20T12:53:53.026 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:53.024+0000 7fbd7a65b700 1 -- 172.21.15.71:0/455536723 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fbd60031090 msgr2=0x7fbd60033550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:53:53.026 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:53.024+0000 7fbd7a65b700 1 --2- 172.21.15.71:0/455536723 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fbd60031090 0x7fbd60033550 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7fbd5c0096a0 tx=0x7fbd5c006b40).stop 2024-02-20T12:53:53.026 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:53.024+0000 7fbd7a65b700 1 -- 172.21.15.71:0/455536723 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbd740f6580 msgr2=0x7fbd7406ba80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:53:53.026 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:53.024+0000 7fbd7a65b700 1 --2- 172.21.15.71:0/455536723 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbd740f6580 0x7fbd7406ba80 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 rx=0x7fbd640089f0 tx=0x7fbd6400b040).stop 2024-02-20T12:53:53.026 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:53.024+0000 7fbd7a65b700 1 -- 172.21.15.71:0/455536723 shutdown_connections 2024-02-20T12:53:53.026 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:53.024+0000 7fbd7a65b700 1 --2- 172.21.15.71:0/455536723 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fbd60031090 0x7fbd60033550 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:53:53.026 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:53.024+0000 7fbd7a65b700 1 --2- 172.21.15.71:0/455536723 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbd740f6580 0x7fbd7406ba80 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:53:53.026 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:53.024+0000 7fbd7a65b700 1 -- 172.21.15.71:0/455536723 >> 172.21.15.71:0/455536723 conn(0x7fbd740f1ce0 msgr2=0x7fbd740f2800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:53:53.027 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:53.024+0000 7fbd7a65b700 1 -- 172.21.15.71:0/455536723 shutdown_connections 2024-02-20T12:53:53.027 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:53.024+0000 7fbd7a65b700 1 -- 172.21.15.71:0/455536723 wait complete. 2024-02-20T12:53:53.027 INFO:teuthology.orchestra.run.smithi071.stderr:dumped monmap epoch 1 2024-02-20T12:53:53.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:53:53 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:53:51.730530+0000 mgr.smithi064.vriyor (mgr.14156) 91 : cluster [DBG] pgmap v83: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:53:53.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:53:53 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:53:53.024025+0000 mon.smithi064 (mon.0) 149 : audit [DBG] from='client.? 172.21.15.71:0/455536723' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-20T12:53:54.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:53:54 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:53:53.845277+0000 mon.smithi064 (mon.0) 150 : audit [INF] from='mgr.14156 172.21.15.64:0/3284974785' entity='mgr.smithi064.vriyor' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi064.vriyor/mirror_snapshot_schedule"}]: dispatch 2024-02-20T12:53:54.563 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:53:54 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:53:53.846117+0000 mon.smithi064 (mon.0) 151 : audit [INF] from='mgr.14156 172.21.15.64:0/3284974785' entity='mgr.smithi064.vriyor' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi064.vriyor/trash_purge_schedule"}]: dispatch 2024-02-20T12:53:54.701 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-20T12:53:54.702 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/daemon-base:latest-pacific shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 680ee126-cfee-11ee-95bb-87774f69a715 -- ceph mon dump -f json 2024-02-20T12:53:55.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:53:55 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:53:53.730807+0000 mgr.smithi064.vriyor (mgr.14156) 92 : cluster [DBG] pgmap v84: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:53:56.527 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:56.488+0000 7f6e529cc700 1 -- 172.21.15.71:0/3118067982 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6e4c0f5480 msgr2=0x7f6e4c0f58a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:53:56.527 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:56.488+0000 7f6e529cc700 1 --2- 172.21.15.71:0/3118067982 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6e4c0f5480 0x7f6e4c0f58a0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 rx=0x7f6e3c009fb0 tx=0x7f6e3c009b30).stop 2024-02-20T12:53:56.527 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:56.488+0000 7f6e529cc700 1 -- 172.21.15.71:0/3118067982 shutdown_connections 2024-02-20T12:53:56.527 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:56.488+0000 7f6e529cc700 1 --2- 172.21.15.71:0/3118067982 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6e4c0f5480 0x7f6e4c0f58a0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:53:56.527 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:56.488+0000 7f6e529cc700 1 -- 172.21.15.71:0/3118067982 >> 172.21.15.71:0/3118067982 conn(0x7f6e4c0f0be0 msgr2=0x7f6e4c0f3000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:53:56.527 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:56.488+0000 7f6e529cc700 1 -- 172.21.15.71:0/3118067982 shutdown_connections 2024-02-20T12:53:56.528 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:56.488+0000 7f6e529cc700 1 -- 172.21.15.71:0/3118067982 wait complete. 2024-02-20T12:53:56.528 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:56.489+0000 7f6e529cc700 1 Processor -- start 2024-02-20T12:53:56.528 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:56.489+0000 7f6e529cc700 1 -- start start 2024-02-20T12:53:56.528 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:56.489+0000 7f6e529cc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6e4c0f5480 0x7f6e4c0fd460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:53:56.528 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:56.489+0000 7f6e529cc700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e4c0fb8e0 con 0x7f6e4c0f5480 2024-02-20T12:53:56.528 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:56.489+0000 7f6e4bfff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6e4c0f5480 0x7f6e4c0fd460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:53:56.528 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:56.489+0000 7f6e4bfff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6e4c0f5480 0x7f6e4c0fd460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.71:40712/0 (socket says 172.21.15.71:40712) 2024-02-20T12:53:56.528 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:56.489+0000 7f6e4bfff700 1 -- 172.21.15.71:0/1832809699 learned_addr learned my addr 172.21.15.71:0/1832809699 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-02-20T12:53:56.528 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:56.489+0000 7f6e4bfff700 1 -- 172.21.15.71:0/1832809699 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e3c005040 con 0x7f6e4c0f5480 2024-02-20T12:53:56.528 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:56.490+0000 7f6e4bfff700 1 --2- 172.21.15.71:0/1832809699 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6e4c0f5480 0x7f6e4c0fd460 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 rx=0x7f6e3c0089f0 tx=0x7f6e3c00b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:53:56.528 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:56.490+0000 7f6e48ff9700 1 -- 172.21.15.71:0/1832809699 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6e3c009320 con 0x7f6e4c0f5480 2024-02-20T12:53:56.528 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:56.490+0000 7f6e529cc700 1 -- 172.21.15.71:0/1832809699 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e4c0fba80 con 0x7f6e4c0f5480 2024-02-20T12:53:56.528 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:56.490+0000 7f6e48ff9700 1 -- 172.21.15.71:0/1832809699 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(25 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f6e3c013ec0 con 0x7f6e4c0f5480 2024-02-20T12:53:56.528 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:56.490+0000 7f6e48ff9700 1 -- 172.21.15.71:0/1832809699 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6e3c014d30 con 0x7f6e4c0f5480 2024-02-20T12:53:56.529 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:56.490+0000 7f6e529cc700 1 -- 172.21.15.71:0/1832809699 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e4c0fbf10 con 0x7f6e4c0f5480 2024-02-20T12:53:56.529 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:56.490+0000 7f6e48ff9700 1 -- 172.21.15.71:0/1832809699 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 37151+0+0 (secure 0 0 0) 0x7f6e3c0095f0 con 0x7f6e4c0f5480 2024-02-20T12:53:56.529 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:56.491+0000 7f6e48ff9700 1 --2- 172.21.15.71:0/1832809699 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f6e38031090 0x7f6e38033550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:53:56.529 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:56.491+0000 7f6e48ff9700 1 -- 172.21.15.71:0/1832809699 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1065+0+0 (secure 0 0 0) 0x7f6e3c0423e0 con 0x7f6e4c0f5480 2024-02-20T12:53:56.529 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:56.491+0000 7f6e529cc700 1 -- 172.21.15.71:0/1832809699 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e4c0591e0 con 0x7f6e4c0f5480 2024-02-20T12:53:56.529 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:56.491+0000 7f6e4b7fe700 1 --2- 172.21.15.71:0/1832809699 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f6e38031090 0x7f6e38033550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:53:56.529 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:56.491+0000 7f6e4b7fe700 1 --2- 172.21.15.71:0/1832809699 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f6e38031090 0x7f6e38033550 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0x7f6e340096a0 tx=0x7f6e34006b40).ready entity=mgr.14156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:53:56.529 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:56.496+0000 7f6e48ff9700 1 -- 172.21.15.71:0/1832809699 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+153237 (secure 0 0 0) 0x7f6e3c01d7f0 con 0x7f6e4c0f5480 2024-02-20T12:53:56.703 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:56.702+0000 7f6e529cc700 1 -- 172.21.15.71:0/1832809699 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f6e4c0faf70 con 0x7f6e4c0f5480 2024-02-20T12:53:56.704 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:56.703+0000 7f6e48ff9700 1 -- 172.21.15.71:0/1832809699 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7f6e3c01d7f0 con 0x7f6e4c0f5480 2024-02-20T12:53:56.704 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-20T12:53:56.705 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":1,"fsid":"680ee126-cfee-11ee-95bb-87774f69a715","modified":"2024-02-20T12:49:44.699059Z","created":"2024-02-20T12:49:44.699059Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi064","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:3300","nonce":0},{"type":"v1","addr":"172.21.15.64:6789","nonce":0}]},"addr":"172.21.15.64:6789/0","public_addr":"172.21.15.64:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-20T12:53:56.705 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:56.704+0000 7f6e529cc700 1 -- 172.21.15.71:0/1832809699 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f6e38031090 msgr2=0x7f6e38033550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:53:56.706 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:56.704+0000 7f6e529cc700 1 --2- 172.21.15.71:0/1832809699 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f6e38031090 0x7f6e38033550 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0x7f6e340096a0 tx=0x7f6e34006b40).stop 2024-02-20T12:53:56.706 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:56.704+0000 7f6e529cc700 1 -- 172.21.15.71:0/1832809699 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6e4c0f5480 msgr2=0x7f6e4c0fd460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:53:56.706 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:56.704+0000 7f6e529cc700 1 --2- 172.21.15.71:0/1832809699 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6e4c0f5480 0x7f6e4c0fd460 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 rx=0x7f6e3c0089f0 tx=0x7f6e3c00b040).stop 2024-02-20T12:53:56.706 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:56.704+0000 7f6e529cc700 1 -- 172.21.15.71:0/1832809699 shutdown_connections 2024-02-20T12:53:56.706 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:56.704+0000 7f6e529cc700 1 --2- 172.21.15.71:0/1832809699 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f6e38031090 0x7f6e38033550 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:53:56.706 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:56.704+0000 7f6e529cc700 1 --2- 172.21.15.71:0/1832809699 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6e4c0f5480 0x7f6e4c0fd460 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:53:56.706 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:56.704+0000 7f6e529cc700 1 -- 172.21.15.71:0/1832809699 >> 172.21.15.71:0/1832809699 conn(0x7f6e4c0f0be0 msgr2=0x7f6e4c05fec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:53:56.706 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:56.704+0000 7f6e529cc700 1 -- 172.21.15.71:0/1832809699 shutdown_connections 2024-02-20T12:53:56.706 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:53:56.704+0000 7f6e529cc700 1 -- 172.21.15.71:0/1832809699 wait complete. 2024-02-20T12:53:56.706 INFO:teuthology.orchestra.run.smithi071.stderr:dumped monmap epoch 1 2024-02-20T12:53:57.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:53:57 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:53:55.731148+0000 mgr.smithi064.vriyor (mgr.14156) 93 : cluster [DBG] pgmap v85: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:53:57.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:53:57 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:53:56.704314+0000 mon.smithi064 (mon.0) 152 : audit [DBG] from='client.? 172.21.15.71:0/1832809699' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-20T12:53:58.389 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-20T12:53:58.389 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/daemon-base:latest-pacific shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 680ee126-cfee-11ee-95bb-87774f69a715 -- ceph mon dump -f json 2024-02-20T12:53:59.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:53:59 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:53:57.731496+0000 mgr.smithi064.vriyor (mgr.14156) 94 : cluster [DBG] pgmap v86: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:54:00.193 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:00.187+0000 7f9a6b716700 1 -- 172.21.15.71:0/341051751 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9a640f5480 msgr2=0x7f9a640f58a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:54:00.193 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:00.187+0000 7f9a6b716700 1 --2- 172.21.15.71:0/341051751 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9a640f5480 0x7f9a640f58a0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 rx=0x7f9a54009fb0 tx=0x7f9a54009b30).stop 2024-02-20T12:54:00.193 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:00.187+0000 7f9a6b716700 1 -- 172.21.15.71:0/341051751 shutdown_connections 2024-02-20T12:54:00.193 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:00.187+0000 7f9a6b716700 1 --2- 172.21.15.71:0/341051751 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9a640f5480 0x7f9a640f58a0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:54:00.193 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:00.187+0000 7f9a6b716700 1 -- 172.21.15.71:0/341051751 >> 172.21.15.71:0/341051751 conn(0x7f9a640f0be0 msgr2=0x7f9a640f3000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:54:00.193 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:00.188+0000 7f9a6b716700 1 -- 172.21.15.71:0/341051751 shutdown_connections 2024-02-20T12:54:00.193 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:00.188+0000 7f9a6b716700 1 -- 172.21.15.71:0/341051751 wait complete. 2024-02-20T12:54:00.194 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:00.188+0000 7f9a6b716700 1 Processor -- start 2024-02-20T12:54:00.194 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:00.188+0000 7f9a6b716700 1 -- start start 2024-02-20T12:54:00.194 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:00.188+0000 7f9a6b716700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9a640f5480 0x7f9a64109510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:54:00.194 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:00.188+0000 7f9a6b716700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a64060300 con 0x7f9a640f5480 2024-02-20T12:54:00.194 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:00.188+0000 7f9a694b2700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9a640f5480 0x7f9a64109510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:54:00.194 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:00.188+0000 7f9a694b2700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9a640f5480 0x7f9a64109510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.71:58292/0 (socket says 172.21.15.71:58292) 2024-02-20T12:54:00.194 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:00.189+0000 7f9a694b2700 1 -- 172.21.15.71:0/3577500128 learned_addr learned my addr 172.21.15.71:0/3577500128 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-02-20T12:54:00.194 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:00.189+0000 7f9a694b2700 1 -- 172.21.15.71:0/3577500128 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a54005040 con 0x7f9a640f5480 2024-02-20T12:54:00.194 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:00.189+0000 7f9a694b2700 1 --2- 172.21.15.71:0/3577500128 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9a640f5480 0x7f9a64109510 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 rx=0x7f9a54000c00 tx=0x7f9a5400b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:54:00.194 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:00.189+0000 7f9a59ffb700 1 -- 172.21.15.71:0/3577500128 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9a540095f0 con 0x7f9a640f5480 2024-02-20T12:54:00.194 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:00.189+0000 7f9a6b716700 1 -- 172.21.15.71:0/3577500128 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a64060500 con 0x7f9a640f5480 2024-02-20T12:54:00.194 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:00.189+0000 7f9a59ffb700 1 -- 172.21.15.71:0/3577500128 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(25 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f9a54007e50 con 0x7f9a640f5480 2024-02-20T12:54:00.194 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:00.189+0000 7f9a59ffb700 1 -- 172.21.15.71:0/3577500128 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9a54014e70 con 0x7f9a640f5480 2024-02-20T12:54:00.194 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:00.189+0000 7f9a6b716700 1 -- 172.21.15.71:0/3577500128 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a64109bc0 con 0x7f9a640f5480 2024-02-20T12:54:00.195 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:00.190+0000 7f9a59ffb700 1 -- 172.21.15.71:0/3577500128 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 37151+0+0 (secure 0 0 0) 0x7f9a54025070 con 0x7f9a640f5480 2024-02-20T12:54:00.195 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:00.190+0000 7f9a6b716700 1 -- 172.21.15.71:0/3577500128 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a640591e0 con 0x7f9a640f5480 2024-02-20T12:54:00.195 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:00.190+0000 7f9a59ffb700 1 --2- 172.21.15.71:0/3577500128 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f9a50031090 0x7f9a50033550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:54:00.195 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:00.190+0000 7f9a59ffb700 1 -- 172.21.15.71:0/3577500128 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1065+0+0 (secure 0 0 0) 0x7f9a54041d20 con 0x7f9a640f5480 2024-02-20T12:54:00.195 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:00.191+0000 7f9a68cb1700 1 --2- 172.21.15.71:0/3577500128 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f9a50031090 0x7f9a50033550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:54:00.195 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:00.191+0000 7f9a68cb1700 1 --2- 172.21.15.71:0/3577500128 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f9a50031090 0x7f9a50033550 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 rx=0x7f9a600096a0 tx=0x7f9a60006b40).ready entity=mgr.14156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:54:00.196 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:00.195+0000 7f9a59ffb700 1 -- 172.21.15.71:0/3577500128 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+153237 (secure 0 0 0) 0x7f9a5400f8d0 con 0x7f9a640f5480 2024-02-20T12:54:00.405 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:00.403+0000 7f9a6b716700 1 -- 172.21.15.71:0/3577500128 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f9a64045bb0 con 0x7f9a640f5480 2024-02-20T12:54:00.405 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:00.404+0000 7f9a59ffb700 1 -- 172.21.15.71:0/3577500128 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7f9a5400f300 con 0x7f9a640f5480 2024-02-20T12:54:00.406 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-20T12:54:00.406 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":1,"fsid":"680ee126-cfee-11ee-95bb-87774f69a715","modified":"2024-02-20T12:49:44.699059Z","created":"2024-02-20T12:49:44.699059Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi064","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:3300","nonce":0},{"type":"v1","addr":"172.21.15.64:6789","nonce":0}]},"addr":"172.21.15.64:6789/0","public_addr":"172.21.15.64:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-20T12:54:00.407 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:00.405+0000 7f9a6b716700 1 -- 172.21.15.71:0/3577500128 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f9a50031090 msgr2=0x7f9a50033550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:54:00.407 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:00.405+0000 7f9a6b716700 1 --2- 172.21.15.71:0/3577500128 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f9a50031090 0x7f9a50033550 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 rx=0x7f9a600096a0 tx=0x7f9a60006b40).stop 2024-02-20T12:54:00.407 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:00.405+0000 7f9a6b716700 1 -- 172.21.15.71:0/3577500128 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9a640f5480 msgr2=0x7f9a64109510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:54:00.407 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:00.405+0000 7f9a6b716700 1 --2- 172.21.15.71:0/3577500128 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9a640f5480 0x7f9a64109510 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 rx=0x7f9a54000c00 tx=0x7f9a5400b040).stop 2024-02-20T12:54:00.407 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:00.405+0000 7f9a6b716700 1 -- 172.21.15.71:0/3577500128 shutdown_connections 2024-02-20T12:54:00.407 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:00.405+0000 7f9a6b716700 1 --2- 172.21.15.71:0/3577500128 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f9a50031090 0x7f9a50033550 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:54:00.407 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:00.405+0000 7f9a6b716700 1 --2- 172.21.15.71:0/3577500128 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9a640f5480 0x7f9a64109510 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:54:00.407 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:00.405+0000 7f9a6b716700 1 -- 172.21.15.71:0/3577500128 >> 172.21.15.71:0/3577500128 conn(0x7f9a640f0be0 msgr2=0x7f9a640f16d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:54:00.408 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:00.405+0000 7f9a6b716700 1 -- 172.21.15.71:0/3577500128 shutdown_connections 2024-02-20T12:54:00.408 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:00.405+0000 7f9a6b716700 1 -- 172.21.15.71:0/3577500128 wait complete. 2024-02-20T12:54:00.408 INFO:teuthology.orchestra.run.smithi071.stderr:dumped monmap epoch 1 2024-02-20T12:54:01.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:54:01 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:53:59.731856+0000 mgr.smithi064.vriyor (mgr.14156) 95 : cluster [DBG] pgmap v87: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:54:01.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:54:01 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:54:00.405590+0000 mon.smithi064 (mon.0) 153 : audit [DBG] from='client.? 172.21.15.71:0/3577500128' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-20T12:54:02.120 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-20T12:54:02.121 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/daemon-base:latest-pacific shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 680ee126-cfee-11ee-95bb-87774f69a715 -- ceph mon dump -f json 2024-02-20T12:54:03.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:54:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:54:01.732265+0000 mgr.smithi064.vriyor (mgr.14156) 96 : cluster [DBG] pgmap v88: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:54:03.898 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:03.895+0000 7f0244c32700 1 -- 172.21.15.71:0/3048497656 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f02400f5480 msgr2=0x7f02400f58a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:54:03.899 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:03.895+0000 7f0244c32700 1 --2- 172.21.15.71:0/3048497656 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f02400f5480 0x7f02400f58a0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 rx=0x7f0228009fb0 tx=0x7f0228009b30).stop 2024-02-20T12:54:03.899 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:03.895+0000 7f0244c32700 1 -- 172.21.15.71:0/3048497656 shutdown_connections 2024-02-20T12:54:03.899 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:03.895+0000 7f0244c32700 1 --2- 172.21.15.71:0/3048497656 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f02400f5480 0x7f02400f58a0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:54:03.899 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:03.895+0000 7f0244c32700 1 -- 172.21.15.71:0/3048497656 >> 172.21.15.71:0/3048497656 conn(0x7f02400f0be0 msgr2=0x7f02400f3000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:54:03.899 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:03.895+0000 7f0244c32700 1 -- 172.21.15.71:0/3048497656 shutdown_connections 2024-02-20T12:54:03.899 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:03.895+0000 7f0244c32700 1 -- 172.21.15.71:0/3048497656 wait complete. 2024-02-20T12:54:03.899 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:03.896+0000 7f0244c32700 1 Processor -- start 2024-02-20T12:54:03.899 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:03.896+0000 7f0244c32700 1 -- start start 2024-02-20T12:54:03.899 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:03.896+0000 7f0244c32700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f02400f5480 0x7f02401056f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:54:03.899 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:03.896+0000 7f0244c32700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0240105be0 con 0x7f02400f5480 2024-02-20T12:54:03.899 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:03.896+0000 7f023e59c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f02400f5480 0x7f02401056f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:54:03.899 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:03.896+0000 7f023e59c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f02400f5480 0x7f02401056f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.71:58308/0 (socket says 172.21.15.71:58308) 2024-02-20T12:54:03.900 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:03.896+0000 7f023e59c700 1 -- 172.21.15.71:0/2756179982 learned_addr learned my addr 172.21.15.71:0/2756179982 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-02-20T12:54:03.900 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:03.896+0000 7f023e59c700 1 -- 172.21.15.71:0/2756179982 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0228005040 con 0x7f02400f5480 2024-02-20T12:54:03.900 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:03.897+0000 7f023e59c700 1 --2- 172.21.15.71:0/2756179982 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f02400f5480 0x7f02401056f0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 rx=0x7f02280089f0 tx=0x7f022800b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:54:03.900 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:03.897+0000 7f0236ffd700 1 -- 172.21.15.71:0/2756179982 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0228009320 con 0x7f02400f5480 2024-02-20T12:54:03.900 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:03.897+0000 7f0244c32700 1 -- 172.21.15.71:0/2756179982 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0240105de0 con 0x7f02400f5480 2024-02-20T12:54:03.900 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:03.897+0000 7f0244c32700 1 -- 172.21.15.71:0/2756179982 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0240106270 con 0x7f02400f5480 2024-02-20T12:54:03.900 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:03.897+0000 7f0236ffd700 1 -- 172.21.15.71:0/2756179982 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(25 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f0228004490 con 0x7f02400f5480 2024-02-20T12:54:03.900 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:03.897+0000 7f0236ffd700 1 -- 172.21.15.71:0/2756179982 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0228014c50 con 0x7f02400f5480 2024-02-20T12:54:03.901 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:03.898+0000 7f0236ffd700 1 -- 172.21.15.71:0/2756179982 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 37151+0+0 (secure 0 0 0) 0x7f0228014420 con 0x7f02400f5480 2024-02-20T12:54:03.901 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:03.898+0000 7f0244c32700 1 -- 172.21.15.71:0/2756179982 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0224004fa0 con 0x7f02400f5480 2024-02-20T12:54:03.901 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:03.898+0000 7f0236ffd700 1 --2- 172.21.15.71:0/2756179982 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f022c031090 0x7f022c033550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:54:03.901 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:03.898+0000 7f0236ffd700 1 -- 172.21.15.71:0/2756179982 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1065+0+0 (secure 0 0 0) 0x7f0228042590 con 0x7f02400f5480 2024-02-20T12:54:03.901 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:03.898+0000 7f023dd9b700 1 --2- 172.21.15.71:0/2756179982 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f022c031090 0x7f022c033550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:54:03.901 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:03.899+0000 7f023dd9b700 1 --2- 172.21.15.71:0/2756179982 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f022c031090 0x7f022c033550 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 rx=0x7f02300096a0 tx=0x7f0230006b40).ready entity=mgr.14156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:54:03.904 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:03.902+0000 7f0236ffd700 1 -- 172.21.15.71:0/2756179982 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+153237 (secure 0 0 0) 0x7f0228025e40 con 0x7f02400f5480 2024-02-20T12:54:04.110 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:04.109+0000 7f0244c32700 1 -- 172.21.15.71:0/2756179982 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f0224005670 con 0x7f02400f5480 2024-02-20T12:54:04.111 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:04.110+0000 7f0236ffd700 1 -- 172.21.15.71:0/2756179982 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7f0228025e40 con 0x7f02400f5480 2024-02-20T12:54:04.111 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-20T12:54:04.111 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":1,"fsid":"680ee126-cfee-11ee-95bb-87774f69a715","modified":"2024-02-20T12:49:44.699059Z","created":"2024-02-20T12:49:44.699059Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi064","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:3300","nonce":0},{"type":"v1","addr":"172.21.15.64:6789","nonce":0}]},"addr":"172.21.15.64:6789/0","public_addr":"172.21.15.64:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-20T12:54:04.113 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:04.111+0000 7f0244c32700 1 -- 172.21.15.71:0/2756179982 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f022c031090 msgr2=0x7f022c033550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:54:04.113 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:04.111+0000 7f0244c32700 1 --2- 172.21.15.71:0/2756179982 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f022c031090 0x7f022c033550 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 rx=0x7f02300096a0 tx=0x7f0230006b40).stop 2024-02-20T12:54:04.113 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:04.111+0000 7f0244c32700 1 -- 172.21.15.71:0/2756179982 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f02400f5480 msgr2=0x7f02401056f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:54:04.113 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:04.111+0000 7f0244c32700 1 --2- 172.21.15.71:0/2756179982 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f02400f5480 0x7f02401056f0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 rx=0x7f02280089f0 tx=0x7f022800b040).stop 2024-02-20T12:54:04.113 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:04.111+0000 7f0244c32700 1 -- 172.21.15.71:0/2756179982 shutdown_connections 2024-02-20T12:54:04.113 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:04.111+0000 7f0244c32700 1 --2- 172.21.15.71:0/2756179982 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f022c031090 0x7f022c033550 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:54:04.113 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:04.111+0000 7f0244c32700 1 --2- 172.21.15.71:0/2756179982 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f02400f5480 0x7f02401056f0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:54:04.113 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:04.111+0000 7f0244c32700 1 -- 172.21.15.71:0/2756179982 >> 172.21.15.71:0/2756179982 conn(0x7f02400f0be0 msgr2=0x7f02400f16d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:54:04.113 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:04.111+0000 7f0244c32700 1 -- 172.21.15.71:0/2756179982 shutdown_connections 2024-02-20T12:54:04.113 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:04.111+0000 7f0244c32700 1 -- 172.21.15.71:0/2756179982 wait complete. 2024-02-20T12:54:04.114 INFO:teuthology.orchestra.run.smithi071.stderr:dumped monmap epoch 1 2024-02-20T12:54:05.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:54:05 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:54:03.732545+0000 mgr.smithi064.vriyor (mgr.14156) 97 : cluster [DBG] pgmap v89: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:54:05.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:54:05 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:54:04.111271+0000 mon.smithi064 (mon.0) 154 : audit [DBG] from='client.? 172.21.15.71:0/2756179982' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-20T12:54:05.762 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-20T12:54:05.762 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/daemon-base:latest-pacific shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 680ee126-cfee-11ee-95bb-87774f69a715 -- ceph mon dump -f json 2024-02-20T12:54:07.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:54:07 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:54:05.732877+0000 mgr.smithi064.vriyor (mgr.14156) 98 : cluster [DBG] pgmap v90: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:54:07.741 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:07.738+0000 7f681f152700 1 -- 172.21.15.71:0/3095687550 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f68180f5480 msgr2=0x7f68180f58a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:54:07.742 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:07.738+0000 7f681f152700 1 --2- 172.21.15.71:0/3095687550 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f68180f5480 0x7f68180f58a0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 rx=0x7f680c009fb0 tx=0x7f680c009b30).stop 2024-02-20T12:54:07.742 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:07.738+0000 7f681f152700 1 -- 172.21.15.71:0/3095687550 shutdown_connections 2024-02-20T12:54:07.742 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:07.738+0000 7f681f152700 1 --2- 172.21.15.71:0/3095687550 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f68180f5480 0x7f68180f58a0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:54:07.742 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:07.738+0000 7f681f152700 1 -- 172.21.15.71:0/3095687550 >> 172.21.15.71:0/3095687550 conn(0x7f68180f0be0 msgr2=0x7f68180f3000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:54:07.742 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:07.738+0000 7f681f152700 1 -- 172.21.15.71:0/3095687550 shutdown_connections 2024-02-20T12:54:07.742 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:07.739+0000 7f681f152700 1 -- 172.21.15.71:0/3095687550 wait complete. 2024-02-20T12:54:07.742 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:07.739+0000 7f681f152700 1 Processor -- start 2024-02-20T12:54:07.742 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:07.739+0000 7f681f152700 1 -- start start 2024-02-20T12:54:07.742 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:07.739+0000 7f681f152700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f68180f5480 0x7f68181094f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:54:07.742 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:07.739+0000 7f681f152700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6818060320 con 0x7f68180f5480 2024-02-20T12:54:07.742 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:07.740+0000 7f681ceee700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f68180f5480 0x7f68181094f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:54:07.742 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:07.740+0000 7f681ceee700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f68180f5480 0x7f68181094f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.71:38192/0 (socket says 172.21.15.71:38192) 2024-02-20T12:54:07.743 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:07.740+0000 7f681ceee700 1 -- 172.21.15.71:0/2933649652 learned_addr learned my addr 172.21.15.71:0/2933649652 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-02-20T12:54:07.743 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:07.740+0000 7f681ceee700 1 -- 172.21.15.71:0/2933649652 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f680c005040 con 0x7f68180f5480 2024-02-20T12:54:07.743 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:07.740+0000 7f681ceee700 1 --2- 172.21.15.71:0/2933649652 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f68180f5480 0x7f68181094f0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 rx=0x7f680c0089f0 tx=0x7f680c00b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:54:07.743 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:07.740+0000 7f68157fa700 1 -- 172.21.15.71:0/2933649652 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f680c009320 con 0x7f68180f5480 2024-02-20T12:54:07.743 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:07.740+0000 7f681f152700 1 -- 172.21.15.71:0/2933649652 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6818060520 con 0x7f68180f5480 2024-02-20T12:54:07.743 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:07.740+0000 7f68157fa700 1 -- 172.21.15.71:0/2933649652 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(25 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f680c013ec0 con 0x7f68180f5480 2024-02-20T12:54:07.743 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:07.740+0000 7f68157fa700 1 -- 172.21.15.71:0/2933649652 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f680c014d30 con 0x7f68180f5480 2024-02-20T12:54:07.743 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:07.740+0000 7f681f152700 1 -- 172.21.15.71:0/2933649652 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6818109ba0 con 0x7f68180f5480 2024-02-20T12:54:07.744 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:07.741+0000 7f68157fa700 1 -- 172.21.15.71:0/2933649652 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 37151+0+0 (secure 0 0 0) 0x7f680c0263a0 con 0x7f68180f5480 2024-02-20T12:54:07.744 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:07.741+0000 7f68157fa700 1 --2- 172.21.15.71:0/2933649652 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f6808031090 0x7f6808033550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:54:07.744 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:07.741+0000 7f68157fa700 1 -- 172.21.15.71:0/2933649652 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1065+0+0 (secure 0 0 0) 0x7f680c042400 con 0x7f68180f5480 2024-02-20T12:54:07.744 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:07.741+0000 7f681f152700 1 -- 172.21.15.71:0/2933649652 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68180591e0 con 0x7f68180f5480 2024-02-20T12:54:07.744 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:07.741+0000 7f6817fff700 1 --2- 172.21.15.71:0/2933649652 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f6808031090 0x7f6808033550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:54:07.744 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:07.742+0000 7f6817fff700 1 --2- 172.21.15.71:0/2933649652 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f6808031090 0x7f6808033550 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 rx=0x7f68040096a0 tx=0x7f6804006b40).ready entity=mgr.14156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:54:07.747 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:07.747+0000 7f68157fa700 1 -- 172.21.15.71:0/2933649652 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+153237 (secure 0 0 0) 0x7f680c0266d0 con 0x7f68180f5480 2024-02-20T12:54:07.954 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:07.953+0000 7f681f152700 1 -- 172.21.15.71:0/2933649652 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f6818045bb0 con 0x7f68180f5480 2024-02-20T12:54:07.954 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:07.953+0000 7f68157fa700 1 -- 172.21.15.71:0/2933649652 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7f680c02b020 con 0x7f68180f5480 2024-02-20T12:54:07.955 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-20T12:54:07.955 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":1,"fsid":"680ee126-cfee-11ee-95bb-87774f69a715","modified":"2024-02-20T12:49:44.699059Z","created":"2024-02-20T12:49:44.699059Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi064","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:3300","nonce":0},{"type":"v1","addr":"172.21.15.64:6789","nonce":0}]},"addr":"172.21.15.64:6789/0","public_addr":"172.21.15.64:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-20T12:54:07.956 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:07.955+0000 7f681f152700 1 -- 172.21.15.71:0/2933649652 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f6808031090 msgr2=0x7f6808033550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:54:07.956 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:07.955+0000 7f681f152700 1 --2- 172.21.15.71:0/2933649652 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f6808031090 0x7f6808033550 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 rx=0x7f68040096a0 tx=0x7f6804006b40).stop 2024-02-20T12:54:07.956 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:07.955+0000 7f681f152700 1 -- 172.21.15.71:0/2933649652 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f68180f5480 msgr2=0x7f68181094f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:54:07.957 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:07.955+0000 7f681f152700 1 --2- 172.21.15.71:0/2933649652 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f68180f5480 0x7f68181094f0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 rx=0x7f680c0089f0 tx=0x7f680c00b040).stop 2024-02-20T12:54:07.957 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:07.955+0000 7f681f152700 1 -- 172.21.15.71:0/2933649652 shutdown_connections 2024-02-20T12:54:07.957 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:07.955+0000 7f681f152700 1 --2- 172.21.15.71:0/2933649652 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f6808031090 0x7f6808033550 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:54:07.957 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:07.955+0000 7f681f152700 1 --2- 172.21.15.71:0/2933649652 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f68180f5480 0x7f68181094f0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:54:07.957 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:07.955+0000 7f681f152700 1 -- 172.21.15.71:0/2933649652 >> 172.21.15.71:0/2933649652 conn(0x7f68180f0be0 msgr2=0x7f68180f16d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:54:07.957 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:07.955+0000 7f681f152700 1 -- 172.21.15.71:0/2933649652 shutdown_connections 2024-02-20T12:54:07.957 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:07.955+0000 7f681f152700 1 -- 172.21.15.71:0/2933649652 wait complete. 2024-02-20T12:54:07.957 INFO:teuthology.orchestra.run.smithi071.stderr:dumped monmap epoch 1 2024-02-20T12:54:08.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:54:08 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:54:07.954873+0000 mon.smithi064 (mon.0) 155 : audit [DBG] from='client.? 172.21.15.71:0/2933649652' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-20T12:54:09.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:54:09 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:54:07.733261+0000 mgr.smithi064.vriyor (mgr.14156) 99 : cluster [DBG] pgmap v91: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:54:09.595 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-20T12:54:09.596 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/daemon-base:latest-pacific shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 680ee126-cfee-11ee-95bb-87774f69a715 -- ceph mon dump -f json 2024-02-20T12:54:11.524 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:11.481+0000 7f0e0d7be700 1 -- 172.21.15.71:0/4238582644 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0e08060360 msgr2=0x7f0e080f9190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:54:11.524 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:11.481+0000 7f0e0d7be700 1 --2- 172.21.15.71:0/4238582644 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0e08060360 0x7f0e080f9190 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 rx=0x7f0dfc009fb0 tx=0x7f0dfc009b30).stop 2024-02-20T12:54:11.524 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:11.481+0000 7f0e0d7be700 1 -- 172.21.15.71:0/4238582644 shutdown_connections 2024-02-20T12:54:11.524 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:11.481+0000 7f0e0d7be700 1 --2- 172.21.15.71:0/4238582644 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0e08060360 0x7f0e080f9190 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:54:11.524 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:11.481+0000 7f0e0d7be700 1 -- 172.21.15.71:0/4238582644 >> 172.21.15.71:0/4238582644 conn(0x7f0e080f0c10 msgr2=0x7f0e080f3030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:54:11.524 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:11.481+0000 7f0e0d7be700 1 -- 172.21.15.71:0/4238582644 shutdown_connections 2024-02-20T12:54:11.525 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:11.481+0000 7f0e0d7be700 1 -- 172.21.15.71:0/4238582644 wait complete. 2024-02-20T12:54:11.525 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:11.482+0000 7f0e0d7be700 1 Processor -- start 2024-02-20T12:54:11.525 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:11.482+0000 7f0e0d7be700 1 -- start start 2024-02-20T12:54:11.525 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:11.482+0000 7f0e0d7be700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0e08060360 0x7f0e080f6c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:54:11.525 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:11.482+0000 7f0e0d7be700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e080f5090 con 0x7f0e08060360 2024-02-20T12:54:11.525 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:11.483+0000 7f0e06ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0e08060360 0x7f0e080f6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:54:11.525 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:11.483+0000 7f0e06ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0e08060360 0x7f0e080f6c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.71:38208/0 (socket says 172.21.15.71:38208) 2024-02-20T12:54:11.525 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:11.483+0000 7f0e06ffd700 1 -- 172.21.15.71:0/2272547247 learned_addr learned my addr 172.21.15.71:0/2272547247 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-02-20T12:54:11.525 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:11.483+0000 7f0e06ffd700 1 -- 172.21.15.71:0/2272547247 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0dfc005040 con 0x7f0e08060360 2024-02-20T12:54:11.525 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:11.483+0000 7f0e06ffd700 1 --2- 172.21.15.71:0/2272547247 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0e08060360 0x7f0e080f6c10 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 rx=0x7f0dfc009f80 tx=0x7f0dfc00b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:54:11.525 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:11.483+0000 7f0def7fe700 1 -- 172.21.15.71:0/2272547247 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0dfc009320 con 0x7f0e08060360 2024-02-20T12:54:11.525 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:11.483+0000 7f0e0d7be700 1 -- 172.21.15.71:0/2272547247 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e080f5230 con 0x7f0e08060360 2024-02-20T12:54:11.525 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:11.483+0000 7f0def7fe700 1 -- 172.21.15.71:0/2272547247 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(25 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f0dfc013680 con 0x7f0e08060360 2024-02-20T12:54:11.526 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:11.483+0000 7f0def7fe700 1 -- 172.21.15.71:0/2272547247 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0dfc01eda0 con 0x7f0e08060360 2024-02-20T12:54:11.526 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:11.483+0000 7f0e0d7be700 1 -- 172.21.15.71:0/2272547247 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e080f56c0 con 0x7f0e08060360 2024-02-20T12:54:11.526 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:11.484+0000 7f0def7fe700 1 -- 172.21.15.71:0/2272547247 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 37151+0+0 (secure 0 0 0) 0x7f0dfc01e420 con 0x7f0e08060360 2024-02-20T12:54:11.526 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:11.484+0000 7f0def7fe700 1 --2- 172.21.15.71:0/2272547247 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f0df0031090 0x7f0df0033550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:54:11.526 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:11.484+0000 7f0e0d7be700 1 -- 172.21.15.71:0/2272547247 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e080591e0 con 0x7f0e08060360 2024-02-20T12:54:11.526 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:11.484+0000 7f0def7fe700 1 -- 172.21.15.71:0/2272547247 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1065+0+0 (secure 0 0 0) 0x7f0dfc042350 con 0x7f0e08060360 2024-02-20T12:54:11.526 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:11.484+0000 7f0e067fc700 1 --2- 172.21.15.71:0/2272547247 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f0df0031090 0x7f0df0033550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:54:11.526 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:11.485+0000 7f0e067fc700 1 --2- 172.21.15.71:0/2272547247 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f0df0031090 0x7f0df0033550 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 rx=0x7f0df80096a0 tx=0x7f0df8006b40).ready entity=mgr.14156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:54:11.526 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:11.489+0000 7f0def7fe700 1 -- 172.21.15.71:0/2272547247 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+153237 (secure 0 0 0) 0x7f0dfc01e700 con 0x7f0e08060360 2024-02-20T12:54:11.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:54:11 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:54:09.733592+0000 mgr.smithi064.vriyor (mgr.14156) 100 : cluster [DBG] pgmap v92: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:54:11.697 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:11.696+0000 7f0e0d7be700 1 -- 172.21.15.71:0/2272547247 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f0e080fc090 con 0x7f0e08060360 2024-02-20T12:54:11.697 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:11.696+0000 7f0def7fe700 1 -- 172.21.15.71:0/2272547247 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7f0dfc01cb10 con 0x7f0e08060360 2024-02-20T12:54:11.697 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-20T12:54:11.698 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":1,"fsid":"680ee126-cfee-11ee-95bb-87774f69a715","modified":"2024-02-20T12:49:44.699059Z","created":"2024-02-20T12:49:44.699059Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi064","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:3300","nonce":0},{"type":"v1","addr":"172.21.15.64:6789","nonce":0}]},"addr":"172.21.15.64:6789/0","public_addr":"172.21.15.64:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-20T12:54:11.698 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:11.698+0000 7f0e0d7be700 1 -- 172.21.15.71:0/2272547247 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f0df0031090 msgr2=0x7f0df0033550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:54:11.699 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:11.698+0000 7f0e0d7be700 1 --2- 172.21.15.71:0/2272547247 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f0df0031090 0x7f0df0033550 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 rx=0x7f0df80096a0 tx=0x7f0df8006b40).stop 2024-02-20T12:54:11.699 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:11.698+0000 7f0e0d7be700 1 -- 172.21.15.71:0/2272547247 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0e08060360 msgr2=0x7f0e080f6c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:54:11.699 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:11.698+0000 7f0e0d7be700 1 --2- 172.21.15.71:0/2272547247 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0e08060360 0x7f0e080f6c10 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 rx=0x7f0dfc009f80 tx=0x7f0dfc00b040).stop 2024-02-20T12:54:11.699 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:11.698+0000 7f0e0d7be700 1 -- 172.21.15.71:0/2272547247 shutdown_connections 2024-02-20T12:54:11.699 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:11.698+0000 7f0e0d7be700 1 --2- 172.21.15.71:0/2272547247 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f0df0031090 0x7f0df0033550 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:54:11.699 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:11.698+0000 7f0e0d7be700 1 --2- 172.21.15.71:0/2272547247 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0e08060360 0x7f0e080f6c10 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:54:11.699 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:11.698+0000 7f0e0d7be700 1 -- 172.21.15.71:0/2272547247 >> 172.21.15.71:0/2272547247 conn(0x7f0e080f0c10 msgr2=0x7f0e080f16a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:54:11.699 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:11.698+0000 7f0e0d7be700 1 -- 172.21.15.71:0/2272547247 shutdown_connections 2024-02-20T12:54:11.699 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:11.698+0000 7f0e0d7be700 1 -- 172.21.15.71:0/2272547247 wait complete. 2024-02-20T12:54:11.700 INFO:teuthology.orchestra.run.smithi071.stderr:dumped monmap epoch 1 2024-02-20T12:54:12.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:54:12 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:54:11.697312+0000 mon.smithi064 (mon.0) 156 : audit [DBG] from='client.? 172.21.15.71:0/2272547247' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-20T12:54:13.397 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-20T12:54:13.397 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/daemon-base:latest-pacific shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 680ee126-cfee-11ee-95bb-87774f69a715 -- ceph mon dump -f json 2024-02-20T12:54:13.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:54:13 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:54:11.733909+0000 mgr.smithi064.vriyor (mgr.14156) 101 : cluster [DBG] pgmap v93: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:54:15.297 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:15.295+0000 7f3ba7741700 1 -- 172.21.15.71:0/2860984525 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3ba00f6a90 msgr2=0x7f3ba00f6eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:54:15.297 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:15.295+0000 7f3ba7741700 1 --2- 172.21.15.71:0/2860984525 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3ba00f6a90 0x7f3ba00f6eb0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 rx=0x7f3b90009fb0 tx=0x7f3b90009b30).stop 2024-02-20T12:54:15.297 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:15.295+0000 7f3ba7741700 1 -- 172.21.15.71:0/2860984525 shutdown_connections 2024-02-20T12:54:15.297 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:15.295+0000 7f3ba7741700 1 --2- 172.21.15.71:0/2860984525 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3ba00f6a90 0x7f3ba00f6eb0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:54:15.298 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:15.295+0000 7f3ba7741700 1 -- 172.21.15.71:0/2860984525 >> 172.21.15.71:0/2860984525 conn(0x7f3ba00f0360 msgr2=0x7f3ba00f2780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:54:15.298 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:15.295+0000 7f3ba7741700 1 -- 172.21.15.71:0/2860984525 shutdown_connections 2024-02-20T12:54:15.298 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:15.295+0000 7f3ba7741700 1 -- 172.21.15.71:0/2860984525 wait complete. 2024-02-20T12:54:15.298 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:15.295+0000 7f3ba7741700 1 Processor -- start 2024-02-20T12:54:15.298 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:15.296+0000 7f3ba7741700 1 -- start start 2024-02-20T12:54:15.298 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:15.296+0000 7f3ba7741700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3ba00f6a90 0x7f3ba010b920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:54:15.298 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:15.296+0000 7f3ba7741700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ba0107310 con 0x7f3ba00f6a90 2024-02-20T12:54:15.298 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:15.296+0000 7f3ba54dd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3ba00f6a90 0x7f3ba010b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:54:15.298 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:15.296+0000 7f3ba54dd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3ba00f6a90 0x7f3ba010b920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.71:38228/0 (socket says 172.21.15.71:38228) 2024-02-20T12:54:15.298 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:15.296+0000 7f3ba54dd700 1 -- 172.21.15.71:0/588404426 learned_addr learned my addr 172.21.15.71:0/588404426 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-02-20T12:54:15.298 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:15.296+0000 7f3ba54dd700 1 -- 172.21.15.71:0/588404426 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b90005040 con 0x7f3ba00f6a90 2024-02-20T12:54:15.298 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:15.296+0000 7f3ba54dd700 1 --2- 172.21.15.71:0/588404426 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3ba00f6a90 0x7f3ba010b920 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 rx=0x7f3b900089f0 tx=0x7f3b9000b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:54:15.298 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:15.297+0000 7f3b95ffb700 1 -- 172.21.15.71:0/588404426 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3b90009320 con 0x7f3ba00f6a90 2024-02-20T12:54:15.298 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:15.297+0000 7f3b95ffb700 1 -- 172.21.15.71:0/588404426 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(25 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f3b90013ec0 con 0x7f3ba00f6a90 2024-02-20T12:54:15.298 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:15.297+0000 7f3ba7741700 1 -- 172.21.15.71:0/588404426 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ba00f53b0 con 0x7f3ba00f6a90 2024-02-20T12:54:15.299 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:15.297+0000 7f3b95ffb700 1 -- 172.21.15.71:0/588404426 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3b90014d30 con 0x7f3ba00f6a90 2024-02-20T12:54:15.299 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:15.297+0000 7f3ba7741700 1 -- 172.21.15.71:0/588404426 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ba010be10 con 0x7f3ba00f6a90 2024-02-20T12:54:15.299 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:15.298+0000 7f3b95ffb700 1 -- 172.21.15.71:0/588404426 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 37151+0+0 (secure 0 0 0) 0x7f3b900263a0 con 0x7f3ba00f6a90 2024-02-20T12:54:15.299 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:15.298+0000 7f3ba7741700 1 -- 172.21.15.71:0/588404426 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ba0045bb0 con 0x7f3ba00f6a90 2024-02-20T12:54:15.300 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:15.298+0000 7f3b95ffb700 1 --2- 172.21.15.71:0/588404426 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f3b8c031090 0x7f3b8c033550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:54:15.300 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:15.298+0000 7f3b95ffb700 1 -- 172.21.15.71:0/588404426 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1065+0+0 (secure 0 0 0) 0x7f3b900423f0 con 0x7f3ba00f6a90 2024-02-20T12:54:15.300 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:15.298+0000 7f3ba4cdc700 1 --2- 172.21.15.71:0/588404426 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f3b8c031090 0x7f3b8c033550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:54:15.300 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:15.298+0000 7f3ba4cdc700 1 --2- 172.21.15.71:0/588404426 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f3b8c031090 0x7f3b8c033550 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 rx=0x7f3b9c0096a0 tx=0x7f3b9c006b40).ready entity=mgr.14156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:54:15.303 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:15.303+0000 7f3b95ffb700 1 -- 172.21.15.71:0/588404426 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+153237 (secure 0 0 0) 0x7f3b90012330 con 0x7f3ba00f6a90 2024-02-20T12:54:15.510 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:15.510+0000 7f3ba7741700 1 -- 172.21.15.71:0/588404426 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f3ba010c5d0 con 0x7f3ba00f6a90 2024-02-20T12:54:15.511 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:15.510+0000 7f3b95ffb700 1 -- 172.21.15.71:0/588404426 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7f3b9001db70 con 0x7f3ba00f6a90 2024-02-20T12:54:15.511 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-20T12:54:15.511 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":1,"fsid":"680ee126-cfee-11ee-95bb-87774f69a715","modified":"2024-02-20T12:49:44.699059Z","created":"2024-02-20T12:49:44.699059Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi064","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:3300","nonce":0},{"type":"v1","addr":"172.21.15.64:6789","nonce":0}]},"addr":"172.21.15.64:6789/0","public_addr":"172.21.15.64:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-20T12:54:15.512 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:15.512+0000 7f3ba7741700 1 -- 172.21.15.71:0/588404426 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f3b8c031090 msgr2=0x7f3b8c033550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:54:15.513 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:15.512+0000 7f3ba7741700 1 --2- 172.21.15.71:0/588404426 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f3b8c031090 0x7f3b8c033550 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 rx=0x7f3b9c0096a0 tx=0x7f3b9c006b40).stop 2024-02-20T12:54:15.513 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:15.512+0000 7f3ba7741700 1 -- 172.21.15.71:0/588404426 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3ba00f6a90 msgr2=0x7f3ba010b920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:54:15.513 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:15.512+0000 7f3ba7741700 1 --2- 172.21.15.71:0/588404426 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3ba00f6a90 0x7f3ba010b920 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 rx=0x7f3b900089f0 tx=0x7f3b9000b040).stop 2024-02-20T12:54:15.513 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:15.512+0000 7f3ba7741700 1 -- 172.21.15.71:0/588404426 shutdown_connections 2024-02-20T12:54:15.513 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:15.512+0000 7f3ba7741700 1 --2- 172.21.15.71:0/588404426 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f3b8c031090 0x7f3b8c033550 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:54:15.513 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:15.512+0000 7f3ba7741700 1 --2- 172.21.15.71:0/588404426 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3ba00f6a90 0x7f3ba010b920 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:54:15.513 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:15.512+0000 7f3ba7741700 1 -- 172.21.15.71:0/588404426 >> 172.21.15.71:0/588404426 conn(0x7f3ba00f0360 msgr2=0x7f3ba00f0e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:54:15.513 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:15.512+0000 7f3ba7741700 1 -- 172.21.15.71:0/588404426 shutdown_connections 2024-02-20T12:54:15.513 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:15.512+0000 7f3ba7741700 1 -- 172.21.15.71:0/588404426 wait complete. 2024-02-20T12:54:15.514 INFO:teuthology.orchestra.run.smithi071.stderr:dumped monmap epoch 1 2024-02-20T12:54:15.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:54:15 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:54:13.734200+0000 mgr.smithi064.vriyor (mgr.14156) 102 : cluster [DBG] pgmap v94: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:54:16.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:54:16 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:54:15.511269+0000 mon.smithi064 (mon.0) 157 : audit [DBG] from='client.? 172.21.15.71:0/588404426' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-20T12:54:17.196 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-20T12:54:17.196 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/daemon-base:latest-pacific shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 680ee126-cfee-11ee-95bb-87774f69a715 -- ceph mon dump -f json 2024-02-20T12:54:17.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:54:17 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:54:15.734542+0000 mgr.smithi064.vriyor (mgr.14156) 103 : cluster [DBG] pgmap v95: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:54:19.097 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:19.097+0000 7f5c9dcb3700 1 -- 172.21.15.71:0/2147164660 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5c980f6900 msgr2=0x7f5c980f6d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:54:19.098 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:19.097+0000 7f5c9dcb3700 1 --2- 172.21.15.71:0/2147164660 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5c980f6900 0x7f5c980f6d20 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 rx=0x7f5c8c009fb0 tx=0x7f5c8c009b30).stop 2024-02-20T12:54:19.098 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:19.097+0000 7f5c9dcb3700 1 -- 172.21.15.71:0/2147164660 shutdown_connections 2024-02-20T12:54:19.098 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:19.097+0000 7f5c9dcb3700 1 --2- 172.21.15.71:0/2147164660 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5c980f6900 0x7f5c980f6d20 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:54:19.098 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:19.097+0000 7f5c9dcb3700 1 -- 172.21.15.71:0/2147164660 >> 172.21.15.71:0/2147164660 conn(0x7f5c980f01d0 msgr2=0x7f5c980f25f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:54:19.098 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:19.097+0000 7f5c9dcb3700 1 -- 172.21.15.71:0/2147164660 shutdown_connections 2024-02-20T12:54:19.098 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:19.097+0000 7f5c9dcb3700 1 -- 172.21.15.71:0/2147164660 wait complete. 2024-02-20T12:54:19.098 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:19.097+0000 7f5c9dcb3700 1 Processor -- start 2024-02-20T12:54:19.098 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:19.097+0000 7f5c9dcb3700 1 -- start start 2024-02-20T12:54:19.100 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:19.097+0000 7f5c9dcb3700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5c980f6900 0x7f5c981092f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:54:19.100 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:19.098+0000 7f5c9dcb3700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c980f5280 con 0x7f5c980f6900 2024-02-20T12:54:19.100 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:19.098+0000 7f5c9ccb1700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5c980f6900 0x7f5c981092f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:54:19.100 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:19.098+0000 7f5c9ccb1700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5c980f6900 0x7f5c981092f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.71:34398/0 (socket says 172.21.15.71:34398) 2024-02-20T12:54:19.100 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:19.098+0000 7f5c9ccb1700 1 -- 172.21.15.71:0/961791240 learned_addr learned my addr 172.21.15.71:0/961791240 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-02-20T12:54:19.100 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:19.098+0000 7f5c9ccb1700 1 -- 172.21.15.71:0/961791240 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c8c005040 con 0x7f5c980f6900 2024-02-20T12:54:19.100 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:19.098+0000 7f5c9ccb1700 1 --2- 172.21.15.71:0/961791240 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5c980f6900 0x7f5c981092f0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 rx=0x7f5c8c0089f0 tx=0x7f5c8c00b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:54:19.100 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:19.099+0000 7f5c957fa700 1 -- 172.21.15.71:0/961791240 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5c8c009320 con 0x7f5c980f6900 2024-02-20T12:54:19.101 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:19.099+0000 7f5c9dcb3700 1 -- 172.21.15.71:0/961791240 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c980f5480 con 0x7f5c980f6900 2024-02-20T12:54:19.101 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:19.099+0000 7f5c957fa700 1 -- 172.21.15.71:0/961791240 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(25 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f5c8c013ec0 con 0x7f5c980f6900 2024-02-20T12:54:19.101 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:19.099+0000 7f5c9dcb3700 1 -- 172.21.15.71:0/961791240 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c98109a20 con 0x7f5c980f6900 2024-02-20T12:54:19.101 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:19.099+0000 7f5c957fa700 1 -- 172.21.15.71:0/961791240 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5c8c014d30 con 0x7f5c980f6900 2024-02-20T12:54:19.101 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:19.100+0000 7f5c957fa700 1 -- 172.21.15.71:0/961791240 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 37151+0+0 (secure 0 0 0) 0x7f5c8c0095f0 con 0x7f5c980f6900 2024-02-20T12:54:19.101 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:19.100+0000 7f5c9dcb3700 1 -- 172.21.15.71:0/961791240 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c98045bb0 con 0x7f5c980f6900 2024-02-20T12:54:19.101 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:19.100+0000 7f5c957fa700 1 --2- 172.21.15.71:0/961791240 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f5c88031090 0x7f5c88033550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:54:19.101 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:19.100+0000 7f5c957fa700 1 -- 172.21.15.71:0/961791240 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1065+0+0 (secure 0 0 0) 0x7f5c8c042440 con 0x7f5c980f6900 2024-02-20T12:54:19.101 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:19.100+0000 7f5c97fff700 1 --2- 172.21.15.71:0/961791240 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f5c88031090 0x7f5c88033550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:54:19.101 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:19.100+0000 7f5c97fff700 1 --2- 172.21.15.71:0/961791240 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f5c88031090 0x7f5c88033550 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 rx=0x7f5c840096a0 tx=0x7f5c84006b40).ready entity=mgr.14156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:54:19.105 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:19.105+0000 7f5c957fa700 1 -- 172.21.15.71:0/961791240 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+153237 (secure 0 0 0) 0x7f5c8c0268c0 con 0x7f5c980f6900 2024-02-20T12:54:19.310 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:19.309+0000 7f5c9dcb3700 1 -- 172.21.15.71:0/961791240 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f5c980fae00 con 0x7f5c980f6900 2024-02-20T12:54:19.310 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:19.310+0000 7f5c957fa700 1 -- 172.21.15.71:0/961791240 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7f5c8c01d7f0 con 0x7f5c980f6900 2024-02-20T12:54:19.311 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-20T12:54:19.311 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":1,"fsid":"680ee126-cfee-11ee-95bb-87774f69a715","modified":"2024-02-20T12:49:44.699059Z","created":"2024-02-20T12:49:44.699059Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi064","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:3300","nonce":0},{"type":"v1","addr":"172.21.15.64:6789","nonce":0}]},"addr":"172.21.15.64:6789/0","public_addr":"172.21.15.64:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-20T12:54:19.312 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:19.312+0000 7f5c9dcb3700 1 -- 172.21.15.71:0/961791240 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f5c88031090 msgr2=0x7f5c88033550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:54:19.312 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:19.312+0000 7f5c9dcb3700 1 --2- 172.21.15.71:0/961791240 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f5c88031090 0x7f5c88033550 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 rx=0x7f5c840096a0 tx=0x7f5c84006b40).stop 2024-02-20T12:54:19.312 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:19.312+0000 7f5c9dcb3700 1 -- 172.21.15.71:0/961791240 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5c980f6900 msgr2=0x7f5c981092f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:54:19.312 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:19.312+0000 7f5c9dcb3700 1 --2- 172.21.15.71:0/961791240 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5c980f6900 0x7f5c981092f0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 rx=0x7f5c8c0089f0 tx=0x7f5c8c00b040).stop 2024-02-20T12:54:19.312 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:19.312+0000 7f5c9dcb3700 1 -- 172.21.15.71:0/961791240 shutdown_connections 2024-02-20T12:54:19.313 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:19.312+0000 7f5c9dcb3700 1 --2- 172.21.15.71:0/961791240 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f5c88031090 0x7f5c88033550 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:54:19.313 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:19.312+0000 7f5c9dcb3700 1 --2- 172.21.15.71:0/961791240 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5c980f6900 0x7f5c981092f0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:54:19.313 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:19.312+0000 7f5c9dcb3700 1 -- 172.21.15.71:0/961791240 >> 172.21.15.71:0/961791240 conn(0x7f5c980f01d0 msgr2=0x7f5c980f0cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:54:19.313 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:19.312+0000 7f5c9dcb3700 1 -- 172.21.15.71:0/961791240 shutdown_connections 2024-02-20T12:54:19.313 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:19.312+0000 7f5c9dcb3700 1 -- 172.21.15.71:0/961791240 wait complete. 2024-02-20T12:54:19.313 INFO:teuthology.orchestra.run.smithi071.stderr:dumped monmap epoch 1 2024-02-20T12:54:19.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:54:19 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:54:17.734916+0000 mgr.smithi064.vriyor (mgr.14156) 104 : cluster [DBG] pgmap v96: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:54:20.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:54:20 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:54:19.310740+0000 mon.smithi064 (mon.0) 158 : audit [DBG] from='client.? 172.21.15.71:0/961791240' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-20T12:54:21.011 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-20T12:54:21.012 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/daemon-base:latest-pacific shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 680ee126-cfee-11ee-95bb-87774f69a715 -- ceph mon dump -f json 2024-02-20T12:54:21.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:54:21 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:54:19.735251+0000 mgr.smithi064.vriyor (mgr.14156) 105 : cluster [DBG] pgmap v97: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:54:22.799 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:22.771+0000 7f99ed4a9700 1 -- 172.21.15.71:0/2846581437 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f99e80f5480 msgr2=0x7f99e80f58a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:54:22.799 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:22.771+0000 7f99ed4a9700 1 --2- 172.21.15.71:0/2846581437 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f99e80f5480 0x7f99e80f58a0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 rx=0x7f99dc009fb0 tx=0x7f99dc009b30).stop 2024-02-20T12:54:22.799 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:22.771+0000 7f99ed4a9700 1 -- 172.21.15.71:0/2846581437 shutdown_connections 2024-02-20T12:54:22.799 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:22.771+0000 7f99ed4a9700 1 --2- 172.21.15.71:0/2846581437 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f99e80f5480 0x7f99e80f58a0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:54:22.799 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:22.771+0000 7f99ed4a9700 1 -- 172.21.15.71:0/2846581437 >> 172.21.15.71:0/2846581437 conn(0x7f99e80f0be0 msgr2=0x7f99e80f3000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:54:22.799 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:22.771+0000 7f99ed4a9700 1 -- 172.21.15.71:0/2846581437 shutdown_connections 2024-02-20T12:54:22.799 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:22.771+0000 7f99ed4a9700 1 -- 172.21.15.71:0/2846581437 wait complete. 2024-02-20T12:54:22.799 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:22.772+0000 7f99ed4a9700 1 Processor -- start 2024-02-20T12:54:22.800 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:22.772+0000 7f99ed4a9700 1 -- start start 2024-02-20T12:54:22.800 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:22.772+0000 7f99ed4a9700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f99e80f5480 0x7f99e80fd460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:54:22.800 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:22.772+0000 7f99ed4a9700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99e80fb8e0 con 0x7f99e80f5480 2024-02-20T12:54:22.800 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:22.772+0000 7f99e6ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f99e80f5480 0x7f99e80fd460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:54:22.800 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:22.772+0000 7f99e6ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f99e80f5480 0x7f99e80fd460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.71:34412/0 (socket says 172.21.15.71:34412) 2024-02-20T12:54:22.800 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:22.772+0000 7f99e6ffd700 1 -- 172.21.15.71:0/3274280884 learned_addr learned my addr 172.21.15.71:0/3274280884 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-02-20T12:54:22.800 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:22.772+0000 7f99e6ffd700 1 -- 172.21.15.71:0/3274280884 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99dc005040 con 0x7f99e80f5480 2024-02-20T12:54:22.800 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:22.773+0000 7f99e6ffd700 1 --2- 172.21.15.71:0/3274280884 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f99e80f5480 0x7f99e80fd460 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 rx=0x7f99dc0089f0 tx=0x7f99dc00b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:54:22.800 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:22.773+0000 7f99cf7fe700 1 -- 172.21.15.71:0/3274280884 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f99dc009320 con 0x7f99e80f5480 2024-02-20T12:54:22.800 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:22.773+0000 7f99ed4a9700 1 -- 172.21.15.71:0/3274280884 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99e80fba80 con 0x7f99e80f5480 2024-02-20T12:54:22.800 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:22.773+0000 7f99cf7fe700 1 -- 172.21.15.71:0/3274280884 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(25 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f99dc013ec0 con 0x7f99e80f5480 2024-02-20T12:54:22.800 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:22.773+0000 7f99cf7fe700 1 -- 172.21.15.71:0/3274280884 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f99dc014d30 con 0x7f99e80f5480 2024-02-20T12:54:22.800 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:22.773+0000 7f99ed4a9700 1 -- 172.21.15.71:0/3274280884 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99e80fbf10 con 0x7f99e80f5480 2024-02-20T12:54:22.800 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:22.774+0000 7f99cf7fe700 1 -- 172.21.15.71:0/3274280884 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 37151+0+0 (secure 0 0 0) 0x7f99dc014420 con 0x7f99e80f5480 2024-02-20T12:54:22.801 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:22.774+0000 7f99ed4a9700 1 -- 172.21.15.71:0/3274280884 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99e80591e0 con 0x7f99e80f5480 2024-02-20T12:54:22.801 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:22.774+0000 7f99cf7fe700 1 --2- 172.21.15.71:0/3274280884 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f99d0031090 0x7f99d0033550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:54:22.801 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:22.774+0000 7f99cf7fe700 1 -- 172.21.15.71:0/3274280884 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1065+0+0 (secure 0 0 0) 0x7f99dc0424a0 con 0x7f99e80f5480 2024-02-20T12:54:22.801 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:22.774+0000 7f99e67fc700 1 --2- 172.21.15.71:0/3274280884 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f99d0031090 0x7f99d0033550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:54:22.801 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:22.774+0000 7f99e67fc700 1 --2- 172.21.15.71:0/3274280884 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f99d0031090 0x7f99d0033550 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 rx=0x7f99d80096a0 tx=0x7f99d8006b40).ready entity=mgr.14156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:54:22.801 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:22.778+0000 7f99cf7fe700 1 -- 172.21.15.71:0/3274280884 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+153237 (secure 0 0 0) 0x7f99dc01d7f0 con 0x7f99e80f5480 2024-02-20T12:54:22.989 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:22.989+0000 7f99ed4a9700 1 -- 172.21.15.71:0/3274280884 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f99e80faf70 con 0x7f99e80f5480 2024-02-20T12:54:22.990 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:22.989+0000 7f99cf7fe700 1 -- 172.21.15.71:0/3274280884 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7f99dc01d7f0 con 0x7f99e80f5480 2024-02-20T12:54:22.990 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-20T12:54:22.990 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":1,"fsid":"680ee126-cfee-11ee-95bb-87774f69a715","modified":"2024-02-20T12:49:44.699059Z","created":"2024-02-20T12:49:44.699059Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi064","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:3300","nonce":0},{"type":"v1","addr":"172.21.15.64:6789","nonce":0}]},"addr":"172.21.15.64:6789/0","public_addr":"172.21.15.64:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-20T12:54:22.991 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:22.991+0000 7f99ed4a9700 1 -- 172.21.15.71:0/3274280884 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f99d0031090 msgr2=0x7f99d0033550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:54:22.991 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:22.991+0000 7f99ed4a9700 1 --2- 172.21.15.71:0/3274280884 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f99d0031090 0x7f99d0033550 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 rx=0x7f99d80096a0 tx=0x7f99d8006b40).stop 2024-02-20T12:54:22.992 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:22.991+0000 7f99ed4a9700 1 -- 172.21.15.71:0/3274280884 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f99e80f5480 msgr2=0x7f99e80fd460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:54:22.992 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:22.991+0000 7f99ed4a9700 1 --2- 172.21.15.71:0/3274280884 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f99e80f5480 0x7f99e80fd460 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 rx=0x7f99dc0089f0 tx=0x7f99dc00b040).stop 2024-02-20T12:54:22.992 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:22.991+0000 7f99ed4a9700 1 -- 172.21.15.71:0/3274280884 shutdown_connections 2024-02-20T12:54:22.992 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:22.991+0000 7f99ed4a9700 1 --2- 172.21.15.71:0/3274280884 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f99d0031090 0x7f99d0033550 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:54:22.992 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:22.991+0000 7f99ed4a9700 1 --2- 172.21.15.71:0/3274280884 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f99e80f5480 0x7f99e80fd460 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:54:22.992 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:22.991+0000 7f99ed4a9700 1 -- 172.21.15.71:0/3274280884 >> 172.21.15.71:0/3274280884 conn(0x7f99e80f0be0 msgr2=0x7f99e805ff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:54:22.992 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:22.991+0000 7f99ed4a9700 1 -- 172.21.15.71:0/3274280884 shutdown_connections 2024-02-20T12:54:22.992 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:22.991+0000 7f99ed4a9700 1 -- 172.21.15.71:0/3274280884 wait complete. 2024-02-20T12:54:22.992 INFO:teuthology.orchestra.run.smithi071.stderr:dumped monmap epoch 1 2024-02-20T12:54:23.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:54:23 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:54:21.735504+0000 mgr.smithi064.vriyor (mgr.14156) 106 : cluster [DBG] pgmap v98: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:54:23.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:54:23 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:54:22.990168+0000 mon.smithi064 (mon.0) 159 : audit [DBG] from='client.? 172.21.15.71:0/3274280884' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-20T12:54:24.688 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-20T12:54:24.688 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/daemon-base:latest-pacific shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 680ee126-cfee-11ee-95bb-87774f69a715 -- ceph mon dump -f json 2024-02-20T12:54:25.563 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:54:25 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:54:23.735845+0000 mgr.smithi064.vriyor (mgr.14156) 107 : cluster [DBG] pgmap v99: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:54:26.954 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:26.928+0000 7fa84cc3b700 1 -- 172.21.15.71:0/2690015681 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa848060360 msgr2=0x7fa8480f8fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:54:26.954 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:26.928+0000 7fa84cc3b700 1 --2- 172.21.15.71:0/2690015681 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa848060360 0x7fa8480f8fb0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 rx=0x7fa838009fb0 tx=0x7fa838009b30).stop 2024-02-20T12:54:26.954 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:26.928+0000 7fa84cc3b700 1 -- 172.21.15.71:0/2690015681 shutdown_connections 2024-02-20T12:54:26.954 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:26.928+0000 7fa84cc3b700 1 --2- 172.21.15.71:0/2690015681 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa848060360 0x7fa8480f8fb0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:54:26.955 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:26.928+0000 7fa84cc3b700 1 -- 172.21.15.71:0/2690015681 >> 172.21.15.71:0/2690015681 conn(0x7fa8480f0a30 msgr2=0x7fa8480f2e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:54:26.955 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:26.928+0000 7fa84cc3b700 1 -- 172.21.15.71:0/2690015681 shutdown_connections 2024-02-20T12:54:26.955 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:26.928+0000 7fa84cc3b700 1 -- 172.21.15.71:0/2690015681 wait complete. 2024-02-20T12:54:26.955 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:26.929+0000 7fa84cc3b700 1 Processor -- start 2024-02-20T12:54:26.955 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:26.929+0000 7fa84cc3b700 1 -- start start 2024-02-20T12:54:26.955 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:26.929+0000 7fa84cc3b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa848060360 0x7fa8480f6a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:54:26.955 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:26.929+0000 7fa84cc3b700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8480f4eb0 con 0x7fa848060360 2024-02-20T12:54:26.955 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:26.929+0000 7fa8477fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa848060360 0x7fa8480f6a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:54:26.955 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:26.929+0000 7fa8477fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa848060360 0x7fa8480f6a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.71:34428/0 (socket says 172.21.15.71:34428) 2024-02-20T12:54:26.955 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:26.929+0000 7fa8477fe700 1 -- 172.21.15.71:0/780574563 learned_addr learned my addr 172.21.15.71:0/780574563 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-02-20T12:54:26.955 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:26.929+0000 7fa8477fe700 1 -- 172.21.15.71:0/780574563 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa838005040 con 0x7fa848060360 2024-02-20T12:54:26.955 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:26.930+0000 7fa8477fe700 1 --2- 172.21.15.71:0/780574563 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa848060360 0x7fa8480f6a30 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 rx=0x7fa838009f80 tx=0x7fa83800b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:54:26.955 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:26.930+0000 7fa82ffff700 1 -- 172.21.15.71:0/780574563 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa838009320 con 0x7fa848060360 2024-02-20T12:54:26.956 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:26.930+0000 7fa82ffff700 1 -- 172.21.15.71:0/780574563 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(25 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7fa838013680 con 0x7fa848060360 2024-02-20T12:54:26.956 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:26.930+0000 7fa82ffff700 1 -- 172.21.15.71:0/780574563 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa83801eda0 con 0x7fa848060360 2024-02-20T12:54:26.956 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:26.930+0000 7fa84cc3b700 1 -- 172.21.15.71:0/780574563 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8480f5050 con 0x7fa848060360 2024-02-20T12:54:26.956 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:26.930+0000 7fa84cc3b700 1 -- 172.21.15.71:0/780574563 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8480f54e0 con 0x7fa848060360 2024-02-20T12:54:26.956 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:26.931+0000 7fa82ffff700 1 -- 172.21.15.71:0/780574563 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 37151+0+0 (secure 0 0 0) 0x7fa83801e420 con 0x7fa848060360 2024-02-20T12:54:26.956 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:26.931+0000 7fa84cc3b700 1 -- 172.21.15.71:0/780574563 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8480591e0 con 0x7fa848060360 2024-02-20T12:54:26.956 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:26.931+0000 7fa82ffff700 1 --2- 172.21.15.71:0/780574563 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fa830031090 0x7fa830033550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:54:26.956 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:26.931+0000 7fa82ffff700 1 -- 172.21.15.71:0/780574563 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1065+0+0 (secure 0 0 0) 0x7fa838042450 con 0x7fa848060360 2024-02-20T12:54:26.956 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:26.931+0000 7fa846ffd700 1 --2- 172.21.15.71:0/780574563 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fa830031090 0x7fa830033550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:54:26.956 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:26.932+0000 7fa846ffd700 1 --2- 172.21.15.71:0/780574563 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fa830031090 0x7fa830033550 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 rx=0x7fa83c0096a0 tx=0x7fa83c006b40).ready entity=mgr.14156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:54:26.956 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:26.936+0000 7fa82ffff700 1 -- 172.21.15.71:0/780574563 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+153237 (secure 0 0 0) 0x7fa83801d380 con 0x7fa848060360 2024-02-20T12:54:27.144 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:27.144+0000 7fa84cc3b700 1 -- 172.21.15.71:0/780574563 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fa8480fbeb0 con 0x7fa848060360 2024-02-20T12:54:27.145 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:27.144+0000 7fa82ffff700 1 -- 172.21.15.71:0/780574563 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7fa838004490 con 0x7fa848060360 2024-02-20T12:54:27.145 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-20T12:54:27.145 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":1,"fsid":"680ee126-cfee-11ee-95bb-87774f69a715","modified":"2024-02-20T12:49:44.699059Z","created":"2024-02-20T12:49:44.699059Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi064","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:3300","nonce":0},{"type":"v1","addr":"172.21.15.64:6789","nonce":0}]},"addr":"172.21.15.64:6789/0","public_addr":"172.21.15.64:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-20T12:54:27.146 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:27.146+0000 7fa84cc3b700 1 -- 172.21.15.71:0/780574563 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fa830031090 msgr2=0x7fa830033550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:54:27.147 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:27.146+0000 7fa84cc3b700 1 --2- 172.21.15.71:0/780574563 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fa830031090 0x7fa830033550 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 rx=0x7fa83c0096a0 tx=0x7fa83c006b40).stop 2024-02-20T12:54:27.147 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:27.146+0000 7fa84cc3b700 1 -- 172.21.15.71:0/780574563 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa848060360 msgr2=0x7fa8480f6a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:54:27.147 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:27.146+0000 7fa84cc3b700 1 --2- 172.21.15.71:0/780574563 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa848060360 0x7fa8480f6a30 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 rx=0x7fa838009f80 tx=0x7fa83800b040).stop 2024-02-20T12:54:27.147 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:27.146+0000 7fa84cc3b700 1 -- 172.21.15.71:0/780574563 shutdown_connections 2024-02-20T12:54:27.147 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:27.146+0000 7fa84cc3b700 1 --2- 172.21.15.71:0/780574563 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fa830031090 0x7fa830033550 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:54:27.147 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:27.146+0000 7fa84cc3b700 1 --2- 172.21.15.71:0/780574563 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa848060360 0x7fa8480f6a30 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:54:27.147 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:27.146+0000 7fa84cc3b700 1 -- 172.21.15.71:0/780574563 >> 172.21.15.71:0/780574563 conn(0x7fa8480f0a30 msgr2=0x7fa8480f14e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:54:27.147 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:27.146+0000 7fa84cc3b700 1 -- 172.21.15.71:0/780574563 shutdown_connections 2024-02-20T12:54:27.147 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:27.146+0000 7fa84cc3b700 1 -- 172.21.15.71:0/780574563 wait complete. 2024-02-20T12:54:27.147 INFO:teuthology.orchestra.run.smithi071.stderr:dumped monmap epoch 1 2024-02-20T12:54:27.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:54:27 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:54:25.736212+0000 mgr.smithi064.vriyor (mgr.14156) 108 : cluster [DBG] pgmap v100: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:54:28.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:54:28 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:54:27.145134+0000 mon.smithi064 (mon.0) 160 : audit [DBG] from='client.? 172.21.15.71:0/780574563' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-20T12:54:28.836 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-20T12:54:28.836 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/daemon-base:latest-pacific shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 680ee126-cfee-11ee-95bb-87774f69a715 -- ceph mon dump -f json 2024-02-20T12:54:29.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:54:29 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:54:27.736593+0000 mgr.smithi064.vriyor (mgr.14156) 109 : cluster [DBG] pgmap v101: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:54:30.822 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:30.820+0000 7f036b8b2700 1 -- 172.21.15.71:0/2213736997 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f03640f5480 msgr2=0x7f03640f58a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:54:30.822 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:30.820+0000 7f036b8b2700 1 --2- 172.21.15.71:0/2213736997 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f03640f5480 0x7f03640f58a0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 rx=0x7f035c009fb0 tx=0x7f035c009b30).stop 2024-02-20T12:54:30.822 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:30.820+0000 7f036b8b2700 1 -- 172.21.15.71:0/2213736997 shutdown_connections 2024-02-20T12:54:30.822 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:30.820+0000 7f036b8b2700 1 --2- 172.21.15.71:0/2213736997 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f03640f5480 0x7f03640f58a0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:54:30.822 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:30.820+0000 7f036b8b2700 1 -- 172.21.15.71:0/2213736997 >> 172.21.15.71:0/2213736997 conn(0x7f03640f0be0 msgr2=0x7f03640f3000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:54:30.822 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:30.820+0000 7f036b8b2700 1 -- 172.21.15.71:0/2213736997 shutdown_connections 2024-02-20T12:54:30.823 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:30.820+0000 7f036b8b2700 1 -- 172.21.15.71:0/2213736997 wait complete. 2024-02-20T12:54:30.823 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:30.820+0000 7f036b8b2700 1 Processor -- start 2024-02-20T12:54:30.823 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:30.821+0000 7f036b8b2700 1 -- start start 2024-02-20T12:54:30.823 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:30.821+0000 7f036b8b2700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f03640f5480 0x7f03641094b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:54:30.823 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:30.821+0000 7f036b8b2700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0364060360 con 0x7f03640f5480 2024-02-20T12:54:30.823 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:30.821+0000 7f036964e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f03640f5480 0x7f03641094b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:54:30.823 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:30.821+0000 7f036964e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f03640f5480 0x7f03641094b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.71:52196/0 (socket says 172.21.15.71:52196) 2024-02-20T12:54:30.823 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:30.821+0000 7f036964e700 1 -- 172.21.15.71:0/3353492780 learned_addr learned my addr 172.21.15.71:0/3353492780 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-02-20T12:54:30.823 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:30.821+0000 7f036964e700 1 -- 172.21.15.71:0/3353492780 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f035c005040 con 0x7f03640f5480 2024-02-20T12:54:30.823 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:30.821+0000 7f036964e700 1 --2- 172.21.15.71:0/3353492780 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f03640f5480 0x7f03641094b0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 rx=0x7f035c0089f0 tx=0x7f035c00b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:54:30.823 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:30.821+0000 7f0361ffb700 1 -- 172.21.15.71:0/3353492780 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f035c009320 con 0x7f03640f5480 2024-02-20T12:54:30.823 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:30.822+0000 7f036b8b2700 1 -- 172.21.15.71:0/3353492780 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0364060560 con 0x7f03640f5480 2024-02-20T12:54:30.823 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:30.822+0000 7f0361ffb700 1 -- 172.21.15.71:0/3353492780 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(25 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f035c013ec0 con 0x7f03640f5480 2024-02-20T12:54:30.823 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:30.822+0000 7f0361ffb700 1 -- 172.21.15.71:0/3353492780 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f035c014d30 con 0x7f03640f5480 2024-02-20T12:54:30.824 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:30.822+0000 7f036b8b2700 1 -- 172.21.15.71:0/3353492780 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0364109be0 con 0x7f03640f5480 2024-02-20T12:54:30.824 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:30.822+0000 7f0361ffb700 1 -- 172.21.15.71:0/3353492780 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 37151+0+0 (secure 0 0 0) 0x7f035c0095f0 con 0x7f03640f5480 2024-02-20T12:54:30.824 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:30.823+0000 7f0361ffb700 1 --2- 172.21.15.71:0/3353492780 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f034c031090 0x7f034c033550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:54:30.825 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:30.823+0000 7f036b8b2700 1 -- 172.21.15.71:0/3353492780 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0364045bb0 con 0x7f03640f5480 2024-02-20T12:54:30.825 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:30.823+0000 7f0361ffb700 1 -- 172.21.15.71:0/3353492780 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1065+0+0 (secure 0 0 0) 0x7f035c0424e0 con 0x7f03640f5480 2024-02-20T12:54:30.825 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:30.823+0000 7f0368e4d700 1 --2- 172.21.15.71:0/3353492780 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f034c031090 0x7f034c033550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:54:30.825 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:30.823+0000 7f0368e4d700 1 --2- 172.21.15.71:0/3353492780 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f034c031090 0x7f034c033550 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 rx=0x7f03580096a0 tx=0x7f0358006b40).ready entity=mgr.14156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:54:30.828 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:30.828+0000 7f0361ffb700 1 -- 172.21.15.71:0/3353492780 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+153237 (secure 0 0 0) 0x7f035c01d7f0 con 0x7f03640f5480 2024-02-20T12:54:31.035 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:31.035+0000 7f036b8b2700 1 -- 172.21.15.71:0/3353492780 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f036410a400 con 0x7f03640f5480 2024-02-20T12:54:31.036 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:31.035+0000 7f0361ffb700 1 -- 172.21.15.71:0/3353492780 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7f035c02c090 con 0x7f03640f5480 2024-02-20T12:54:31.036 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-20T12:54:31.036 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":1,"fsid":"680ee126-cfee-11ee-95bb-87774f69a715","modified":"2024-02-20T12:49:44.699059Z","created":"2024-02-20T12:49:44.699059Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi064","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:3300","nonce":0},{"type":"v1","addr":"172.21.15.64:6789","nonce":0}]},"addr":"172.21.15.64:6789/0","public_addr":"172.21.15.64:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-20T12:54:31.037 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:31.037+0000 7f036b8b2700 1 -- 172.21.15.71:0/3353492780 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f034c031090 msgr2=0x7f034c033550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:54:31.038 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:31.037+0000 7f036b8b2700 1 --2- 172.21.15.71:0/3353492780 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f034c031090 0x7f034c033550 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 rx=0x7f03580096a0 tx=0x7f0358006b40).stop 2024-02-20T12:54:31.038 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:31.037+0000 7f036b8b2700 1 -- 172.21.15.71:0/3353492780 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f03640f5480 msgr2=0x7f03641094b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:54:31.038 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:31.037+0000 7f036b8b2700 1 --2- 172.21.15.71:0/3353492780 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f03640f5480 0x7f03641094b0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 rx=0x7f035c0089f0 tx=0x7f035c00b040).stop 2024-02-20T12:54:31.038 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:31.037+0000 7f036b8b2700 1 -- 172.21.15.71:0/3353492780 shutdown_connections 2024-02-20T12:54:31.038 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:31.037+0000 7f036b8b2700 1 --2- 172.21.15.71:0/3353492780 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f034c031090 0x7f034c033550 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:54:31.038 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:31.037+0000 7f036b8b2700 1 --2- 172.21.15.71:0/3353492780 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f03640f5480 0x7f03641094b0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:54:31.038 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:31.037+0000 7f036b8b2700 1 -- 172.21.15.71:0/3353492780 >> 172.21.15.71:0/3353492780 conn(0x7f03640f0be0 msgr2=0x7f03640f16d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:54:31.038 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:31.037+0000 7f036b8b2700 1 -- 172.21.15.71:0/3353492780 shutdown_connections 2024-02-20T12:54:31.038 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:31.037+0000 7f036b8b2700 1 -- 172.21.15.71:0/3353492780 wait complete. 2024-02-20T12:54:31.038 INFO:teuthology.orchestra.run.smithi071.stderr:dumped monmap epoch 1 2024-02-20T12:54:31.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:54:31 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:54:29.736905+0000 mgr.smithi064.vriyor (mgr.14156) 110 : cluster [DBG] pgmap v102: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:54:31.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:54:31 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:54:31.036179+0000 mon.smithi064 (mon.0) 161 : audit [DBG] from='client.? 172.21.15.71:0/3353492780' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-20T12:54:32.677 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-20T12:54:32.678 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/daemon-base:latest-pacific shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 680ee126-cfee-11ee-95bb-87774f69a715 -- ceph mon dump -f json 2024-02-20T12:54:33.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:54:33 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:54:31.737251+0000 mgr.smithi064.vriyor (mgr.14156) 111 : cluster [DBG] pgmap v103: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:54:34.597 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:34.595+0000 7f37d1a76700 1 -- 172.21.15.71:0/2412326040 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f37cc060360 msgr2=0x7f37cc0f9160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:54:34.597 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:34.595+0000 7f37d1a76700 1 --2- 172.21.15.71:0/2412326040 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f37cc060360 0x7f37cc0f9160 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 rx=0x7f37b4009fb0 tx=0x7f37b4009b30).stop 2024-02-20T12:54:34.598 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:34.595+0000 7f37d1a76700 1 -- 172.21.15.71:0/2412326040 shutdown_connections 2024-02-20T12:54:34.598 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:34.595+0000 7f37d1a76700 1 --2- 172.21.15.71:0/2412326040 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f37cc060360 0x7f37cc0f9160 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:54:34.598 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:34.595+0000 7f37d1a76700 1 -- 172.21.15.71:0/2412326040 >> 172.21.15.71:0/2412326040 conn(0x7f37cc0f0be0 msgr2=0x7f37cc0f3000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:54:34.598 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:34.595+0000 7f37d1a76700 1 -- 172.21.15.71:0/2412326040 shutdown_connections 2024-02-20T12:54:34.598 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:34.595+0000 7f37d1a76700 1 -- 172.21.15.71:0/2412326040 wait complete. 2024-02-20T12:54:34.598 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:34.596+0000 7f37d1a76700 1 Processor -- start 2024-02-20T12:54:34.598 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:34.596+0000 7f37d1a76700 1 -- start start 2024-02-20T12:54:34.598 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:34.596+0000 7f37d1a76700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f37cc060360 0x7f37cc0f6be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:54:34.598 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:34.596+0000 7f37d1a76700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37cc0f5060 con 0x7f37cc060360 2024-02-20T12:54:34.598 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:34.596+0000 7f37cb7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f37cc060360 0x7f37cc0f6be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:54:34.598 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:34.596+0000 7f37cb7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f37cc060360 0x7f37cc0f6be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.71:52210/0 (socket says 172.21.15.71:52210) 2024-02-20T12:54:34.598 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:34.596+0000 7f37cb7fe700 1 -- 172.21.15.71:0/1590352628 learned_addr learned my addr 172.21.15.71:0/1590352628 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-02-20T12:54:34.598 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:34.597+0000 7f37cb7fe700 1 -- 172.21.15.71:0/1590352628 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37b4005040 con 0x7f37cc060360 2024-02-20T12:54:34.598 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:34.597+0000 7f37cb7fe700 1 --2- 172.21.15.71:0/1590352628 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f37cc060360 0x7f37cc0f6be0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 rx=0x7f37b4009f80 tx=0x7f37b400b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:54:34.598 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:34.597+0000 7f37c3fff700 1 -- 172.21.15.71:0/1590352628 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f37b4009320 con 0x7f37cc060360 2024-02-20T12:54:34.599 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:34.597+0000 7f37c3fff700 1 -- 172.21.15.71:0/1590352628 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(25 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f37b4013680 con 0x7f37cc060360 2024-02-20T12:54:34.599 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:34.597+0000 7f37d1a76700 1 -- 172.21.15.71:0/1590352628 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37cc0f5200 con 0x7f37cc060360 2024-02-20T12:54:34.599 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:34.597+0000 7f37c3fff700 1 -- 172.21.15.71:0/1590352628 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f37b401eda0 con 0x7f37cc060360 2024-02-20T12:54:34.599 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:34.597+0000 7f37d1a76700 1 -- 172.21.15.71:0/1590352628 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37cc0f5690 con 0x7f37cc060360 2024-02-20T12:54:34.599 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:34.598+0000 7f37c3fff700 1 -- 172.21.15.71:0/1590352628 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 37151+0+0 (secure 0 0 0) 0x7f37b401e420 con 0x7f37cc060360 2024-02-20T12:54:34.600 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:34.598+0000 7f37c3fff700 1 --2- 172.21.15.71:0/1590352628 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f37b8031090 0x7f37b8033550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:54:34.600 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:34.598+0000 7f37c3fff700 1 -- 172.21.15.71:0/1590352628 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1065+0+0 (secure 0 0 0) 0x7f37b4042380 con 0x7f37cc060360 2024-02-20T12:54:34.600 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:34.598+0000 7f37d1a76700 1 -- 172.21.15.71:0/1590352628 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37cc0591e0 con 0x7f37cc060360 2024-02-20T12:54:34.600 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:34.598+0000 7f37caffd700 1 --2- 172.21.15.71:0/1590352628 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f37b8031090 0x7f37b8033550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:54:34.600 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:34.599+0000 7f37caffd700 1 --2- 172.21.15.71:0/1590352628 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f37b8031090 0x7f37b8033550 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 rx=0x7f37bc0096a0 tx=0x7f37bc006b40).ready entity=mgr.14156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:54:34.603 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:34.603+0000 7f37c3fff700 1 -- 172.21.15.71:0/1590352628 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+153237 (secure 0 0 0) 0x7f37b401e700 con 0x7f37cc060360 2024-02-20T12:54:34.810 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:34.810+0000 7f37d1a76700 1 -- 172.21.15.71:0/1590352628 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f37cc0fc060 con 0x7f37cc060360 2024-02-20T12:54:34.811 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:34.810+0000 7f37c3fff700 1 -- 172.21.15.71:0/1590352628 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7f37b401cb10 con 0x7f37cc060360 2024-02-20T12:54:34.811 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-20T12:54:34.811 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":1,"fsid":"680ee126-cfee-11ee-95bb-87774f69a715","modified":"2024-02-20T12:49:44.699059Z","created":"2024-02-20T12:49:44.699059Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi064","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:3300","nonce":0},{"type":"v1","addr":"172.21.15.64:6789","nonce":0}]},"addr":"172.21.15.64:6789/0","public_addr":"172.21.15.64:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-20T12:54:34.812 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:34.812+0000 7f37d1a76700 1 -- 172.21.15.71:0/1590352628 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f37b8031090 msgr2=0x7f37b8033550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:54:34.813 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:34.812+0000 7f37d1a76700 1 --2- 172.21.15.71:0/1590352628 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f37b8031090 0x7f37b8033550 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 rx=0x7f37bc0096a0 tx=0x7f37bc006b40).stop 2024-02-20T12:54:34.813 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:34.812+0000 7f37d1a76700 1 -- 172.21.15.71:0/1590352628 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f37cc060360 msgr2=0x7f37cc0f6be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:54:34.813 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:34.812+0000 7f37d1a76700 1 --2- 172.21.15.71:0/1590352628 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f37cc060360 0x7f37cc0f6be0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 rx=0x7f37b4009f80 tx=0x7f37b400b040).stop 2024-02-20T12:54:34.813 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:34.812+0000 7f37d1a76700 1 -- 172.21.15.71:0/1590352628 shutdown_connections 2024-02-20T12:54:34.813 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:34.812+0000 7f37d1a76700 1 --2- 172.21.15.71:0/1590352628 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f37b8031090 0x7f37b8033550 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:54:34.813 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:34.812+0000 7f37d1a76700 1 --2- 172.21.15.71:0/1590352628 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f37cc060360 0x7f37cc0f6be0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:54:34.813 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:34.812+0000 7f37d1a76700 1 -- 172.21.15.71:0/1590352628 >> 172.21.15.71:0/1590352628 conn(0x7f37cc0f0be0 msgr2=0x7f37cc0f1660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:54:34.813 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:34.812+0000 7f37d1a76700 1 -- 172.21.15.71:0/1590352628 shutdown_connections 2024-02-20T12:54:34.813 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:34.812+0000 7f37d1a76700 1 -- 172.21.15.71:0/1590352628 wait complete. 2024-02-20T12:54:34.813 INFO:teuthology.orchestra.run.smithi071.stderr:dumped monmap epoch 1 2024-02-20T12:54:35.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:54:35 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:54:33.737622+0000 mgr.smithi064.vriyor (mgr.14156) 112 : cluster [DBG] pgmap v104: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:54:35.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:54:35 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:54:34.811155+0000 mon.smithi064 (mon.0) 162 : audit [DBG] from='client.? 172.21.15.71:0/1590352628' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-20T12:54:36.536 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-20T12:54:36.536 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/daemon-base:latest-pacific shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 680ee126-cfee-11ee-95bb-87774f69a715 -- ceph mon dump -f json 2024-02-20T12:54:37.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:54:37 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:54:35.738021+0000 mgr.smithi064.vriyor (mgr.14156) 113 : cluster [DBG] pgmap v105: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:54:38.508 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:38.505+0000 7f4892209700 1 -- 172.21.15.71:0/3444026610 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f488c060300 msgr2=0x7f488c0f91c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:54:38.508 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:38.505+0000 7f4892209700 1 --2- 172.21.15.71:0/3444026610 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f488c060300 0x7f488c0f91c0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 rx=0x7f487c009fb0 tx=0x7f487c009b30).stop 2024-02-20T12:54:38.508 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:38.505+0000 7f4892209700 1 -- 172.21.15.71:0/3444026610 shutdown_connections 2024-02-20T12:54:38.508 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:38.505+0000 7f4892209700 1 --2- 172.21.15.71:0/3444026610 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f488c060300 0x7f488c0f91c0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:54:38.508 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:38.505+0000 7f4892209700 1 -- 172.21.15.71:0/3444026610 >> 172.21.15.71:0/3444026610 conn(0x7f488c0f0be0 msgr2=0x7f488c0f3000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:54:38.509 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:38.505+0000 7f4892209700 1 -- 172.21.15.71:0/3444026610 shutdown_connections 2024-02-20T12:54:38.509 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:38.505+0000 7f4892209700 1 -- 172.21.15.71:0/3444026610 wait complete. 2024-02-20T12:54:38.509 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:38.505+0000 7f4892209700 1 Processor -- start 2024-02-20T12:54:38.509 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:38.505+0000 7f4892209700 1 -- start start 2024-02-20T12:54:38.509 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:38.506+0000 7f4892209700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f488c060300 0x7f488c106780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:54:38.509 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:38.506+0000 7f4892209700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f488c106c70 con 0x7f488c060300 2024-02-20T12:54:38.509 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:38.506+0000 7f488b7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f488c060300 0x7f488c106780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:54:38.509 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:38.506+0000 7f488b7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f488c060300 0x7f488c106780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.71:44874/0 (socket says 172.21.15.71:44874) 2024-02-20T12:54:38.509 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:38.506+0000 7f488b7fe700 1 -- 172.21.15.71:0/2737819936 learned_addr learned my addr 172.21.15.71:0/2737819936 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-02-20T12:54:38.509 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:38.506+0000 7f488b7fe700 1 -- 172.21.15.71:0/2737819936 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f487c005040 con 0x7f488c060300 2024-02-20T12:54:38.509 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:38.506+0000 7f488b7fe700 1 --2- 172.21.15.71:0/2737819936 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f488c060300 0x7f488c106780 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 rx=0x7f487c009f80 tx=0x7f487c00b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:54:38.509 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:38.506+0000 7f4888ff9700 1 -- 172.21.15.71:0/2737819936 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f487c009320 con 0x7f488c060300 2024-02-20T12:54:38.509 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:38.506+0000 7f4888ff9700 1 -- 172.21.15.71:0/2737819936 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(25 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f487c013680 con 0x7f488c060300 2024-02-20T12:54:38.509 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:38.506+0000 7f4892209700 1 -- 172.21.15.71:0/2737819936 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f488c106e70 con 0x7f488c060300 2024-02-20T12:54:38.509 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:38.506+0000 7f4888ff9700 1 -- 172.21.15.71:0/2737819936 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f487c01eda0 con 0x7f488c060300 2024-02-20T12:54:38.510 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:38.507+0000 7f4892209700 1 -- 172.21.15.71:0/2737819936 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f488c107300 con 0x7f488c060300 2024-02-20T12:54:38.510 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:38.507+0000 7f4888ff9700 1 -- 172.21.15.71:0/2737819936 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 37151+0+0 (secure 0 0 0) 0x7f487c01e420 con 0x7f488c060300 2024-02-20T12:54:38.510 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:38.507+0000 7f4888ff9700 1 --2- 172.21.15.71:0/2737819936 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f4878031090 0x7f4878033550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:54:38.510 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:38.508+0000 7f4888ff9700 1 -- 172.21.15.71:0/2737819936 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1065+0+0 (secure 0 0 0) 0x7f487c042450 con 0x7f488c060300 2024-02-20T12:54:38.510 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:38.508+0000 7f4892209700 1 -- 172.21.15.71:0/2737819936 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f488c045bb0 con 0x7f488c060300 2024-02-20T12:54:38.510 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:38.508+0000 7f4882ffd700 1 --2- 172.21.15.71:0/2737819936 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f4878031090 0x7f4878033550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:54:38.510 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:38.508+0000 7f4882ffd700 1 --2- 172.21.15.71:0/2737819936 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f4878031090 0x7f4878033550 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 rx=0x7f48740096a0 tx=0x7f4874006b40).ready entity=mgr.14156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:54:38.512 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:38.512+0000 7f4888ff9700 1 -- 172.21.15.71:0/2737819936 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+153237 (secure 0 0 0) 0x7f487c01e700 con 0x7f488c060300 2024-02-20T12:54:38.719 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:38.719+0000 7f4892209700 1 -- 172.21.15.71:0/2737819936 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f488c107f80 con 0x7f488c060300 2024-02-20T12:54:38.720 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:38.719+0000 7f4888ff9700 1 -- 172.21.15.71:0/2737819936 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7f487c01cb10 con 0x7f488c060300 2024-02-20T12:54:38.720 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-20T12:54:38.720 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":1,"fsid":"680ee126-cfee-11ee-95bb-87774f69a715","modified":"2024-02-20T12:49:44.699059Z","created":"2024-02-20T12:49:44.699059Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi064","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:3300","nonce":0},{"type":"v1","addr":"172.21.15.64:6789","nonce":0}]},"addr":"172.21.15.64:6789/0","public_addr":"172.21.15.64:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-20T12:54:38.722 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:38.721+0000 7f4892209700 1 -- 172.21.15.71:0/2737819936 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f4878031090 msgr2=0x7f4878033550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:54:38.722 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:38.721+0000 7f4892209700 1 --2- 172.21.15.71:0/2737819936 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f4878031090 0x7f4878033550 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 rx=0x7f48740096a0 tx=0x7f4874006b40).stop 2024-02-20T12:54:38.722 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:38.721+0000 7f4892209700 1 -- 172.21.15.71:0/2737819936 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f488c060300 msgr2=0x7f488c106780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:54:38.722 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:38.721+0000 7f4892209700 1 --2- 172.21.15.71:0/2737819936 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f488c060300 0x7f488c106780 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 rx=0x7f487c009f80 tx=0x7f487c00b040).stop 2024-02-20T12:54:38.722 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:38.721+0000 7f4892209700 1 -- 172.21.15.71:0/2737819936 shutdown_connections 2024-02-20T12:54:38.722 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:38.721+0000 7f4892209700 1 --2- 172.21.15.71:0/2737819936 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f4878031090 0x7f4878033550 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:54:38.722 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:38.721+0000 7f4892209700 1 --2- 172.21.15.71:0/2737819936 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f488c060300 0x7f488c106780 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:54:38.722 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:38.721+0000 7f4892209700 1 -- 172.21.15.71:0/2737819936 >> 172.21.15.71:0/2737819936 conn(0x7f488c0f0be0 msgr2=0x7f488c0f1720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:54:38.722 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:38.721+0000 7f4892209700 1 -- 172.21.15.71:0/2737819936 shutdown_connections 2024-02-20T12:54:38.722 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:38.721+0000 7f4892209700 1 -- 172.21.15.71:0/2737819936 wait complete. 2024-02-20T12:54:38.723 INFO:teuthology.orchestra.run.smithi071.stderr:dumped monmap epoch 1 2024-02-20T12:54:39.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:54:39 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:54:37.738228+0000 mgr.smithi064.vriyor (mgr.14156) 114 : cluster [DBG] pgmap v106: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:54:39.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:54:39 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:54:38.720240+0000 mon.smithi064 (mon.0) 163 : audit [DBG] from='client.? 172.21.15.71:0/2737819936' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-20T12:54:40.320 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-20T12:54:40.321 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/daemon-base:latest-pacific shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 680ee126-cfee-11ee-95bb-87774f69a715 -- ceph mon dump -f json 2024-02-20T12:54:41.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:54:41 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:54:39.738548+0000 mgr.smithi064.vriyor (mgr.14156) 115 : cluster [DBG] pgmap v107: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:54:42.208 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:42.203+0000 7f7948b98700 1 -- 172.21.15.71:0/2301580843 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f79440f5480 msgr2=0x7f79440f58a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:54:42.208 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:42.203+0000 7f7948b98700 1 --2- 172.21.15.71:0/2301580843 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f79440f5480 0x7f79440f58a0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 rx=0x7f792c009fb0 tx=0x7f792c009b30).stop 2024-02-20T12:54:42.208 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:42.204+0000 7f7948b98700 1 -- 172.21.15.71:0/2301580843 shutdown_connections 2024-02-20T12:54:42.208 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:42.204+0000 7f7948b98700 1 --2- 172.21.15.71:0/2301580843 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f79440f5480 0x7f79440f58a0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:54:42.208 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:42.204+0000 7f7948b98700 1 -- 172.21.15.71:0/2301580843 >> 172.21.15.71:0/2301580843 conn(0x7f79440f0be0 msgr2=0x7f79440f3000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:54:42.208 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:42.204+0000 7f7948b98700 1 -- 172.21.15.71:0/2301580843 shutdown_connections 2024-02-20T12:54:42.208 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:42.204+0000 7f7948b98700 1 -- 172.21.15.71:0/2301580843 wait complete. 2024-02-20T12:54:42.208 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:42.204+0000 7f7948b98700 1 Processor -- start 2024-02-20T12:54:42.209 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:42.204+0000 7f7948b98700 1 -- start start 2024-02-20T12:54:42.209 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:42.204+0000 7f7948b98700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f79440f5480 0x7f79441067c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:54:42.209 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:42.204+0000 7f7948b98700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7944106cb0 con 0x7f79440f5480 2024-02-20T12:54:42.209 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:42.205+0000 7f794259c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f79440f5480 0x7f79441067c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:54:42.209 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:42.205+0000 7f794259c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f79440f5480 0x7f79441067c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.71:44882/0 (socket says 172.21.15.71:44882) 2024-02-20T12:54:42.209 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:42.205+0000 7f794259c700 1 -- 172.21.15.71:0/1979379804 learned_addr learned my addr 172.21.15.71:0/1979379804 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-02-20T12:54:42.209 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:42.205+0000 7f794259c700 1 -- 172.21.15.71:0/1979379804 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f792c005040 con 0x7f79440f5480 2024-02-20T12:54:42.209 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:42.205+0000 7f794259c700 1 --2- 172.21.15.71:0/1979379804 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f79440f5480 0x7f79441067c0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 rx=0x7f792c0089f0 tx=0x7f792c00b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:54:42.209 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:42.205+0000 7f793affd700 1 -- 172.21.15.71:0/1979379804 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f792c009320 con 0x7f79440f5480 2024-02-20T12:54:42.209 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:42.205+0000 7f793affd700 1 -- 172.21.15.71:0/1979379804 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(25 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f792c013ec0 con 0x7f79440f5480 2024-02-20T12:54:42.209 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:42.205+0000 7f7948b98700 1 -- 172.21.15.71:0/1979379804 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7944106eb0 con 0x7f79440f5480 2024-02-20T12:54:42.209 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:42.205+0000 7f793affd700 1 -- 172.21.15.71:0/1979379804 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f792c014d30 con 0x7f79440f5480 2024-02-20T12:54:42.209 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:42.205+0000 7f7948b98700 1 -- 172.21.15.71:0/1979379804 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7944107340 con 0x7f79440f5480 2024-02-20T12:54:42.209 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:42.206+0000 7f793affd700 1 -- 172.21.15.71:0/1979379804 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 37151+0+0 (secure 0 0 0) 0x7f792c014420 con 0x7f79440f5480 2024-02-20T12:54:42.210 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:42.206+0000 7f793affd700 1 --2- 172.21.15.71:0/1979379804 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f79300310e0 0x7f79300335a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:54:42.210 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:42.206+0000 7f793affd700 1 -- 172.21.15.71:0/1979379804 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1065+0+0 (secure 0 0 0) 0x7f792c0423b0 con 0x7f79440f5480 2024-02-20T12:54:42.210 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:42.206+0000 7f7948b98700 1 -- 172.21.15.71:0/1979379804 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7944045bb0 con 0x7f79440f5480 2024-02-20T12:54:42.210 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:42.207+0000 7f7941d9b700 1 --2- 172.21.15.71:0/1979379804 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f79300310e0 0x7f79300335a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:54:42.210 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:42.207+0000 7f7941d9b700 1 --2- 172.21.15.71:0/1979379804 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f79300310e0 0x7f79300335a0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 rx=0x7f79340096a0 tx=0x7f7934006b40).ready entity=mgr.14156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:54:42.213 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:42.211+0000 7f793affd700 1 -- 172.21.15.71:0/1979379804 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+153237 (secure 0 0 0) 0x7f792c014750 con 0x7f79440f5480 2024-02-20T12:54:42.418 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:42.416+0000 7f7948b98700 1 -- 172.21.15.71:0/1979379804 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f7944108140 con 0x7f79440f5480 2024-02-20T12:54:42.418 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:42.417+0000 7f793affd700 1 -- 172.21.15.71:0/1979379804 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7f792c01d380 con 0x7f79440f5480 2024-02-20T12:54:42.419 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-20T12:54:42.419 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":1,"fsid":"680ee126-cfee-11ee-95bb-87774f69a715","modified":"2024-02-20T12:49:44.699059Z","created":"2024-02-20T12:49:44.699059Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi064","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:3300","nonce":0},{"type":"v1","addr":"172.21.15.64:6789","nonce":0}]},"addr":"172.21.15.64:6789/0","public_addr":"172.21.15.64:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-20T12:54:42.420 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:42.418+0000 7f7948b98700 1 -- 172.21.15.71:0/1979379804 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f79300310e0 msgr2=0x7f79300335a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:54:42.420 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:42.418+0000 7f7948b98700 1 --2- 172.21.15.71:0/1979379804 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f79300310e0 0x7f79300335a0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 rx=0x7f79340096a0 tx=0x7f7934006b40).stop 2024-02-20T12:54:42.420 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:42.418+0000 7f7948b98700 1 -- 172.21.15.71:0/1979379804 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f79440f5480 msgr2=0x7f79441067c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:54:42.420 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:42.418+0000 7f7948b98700 1 --2- 172.21.15.71:0/1979379804 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f79440f5480 0x7f79441067c0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 rx=0x7f792c0089f0 tx=0x7f792c00b040).stop 2024-02-20T12:54:42.421 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:42.418+0000 7f7948b98700 1 -- 172.21.15.71:0/1979379804 shutdown_connections 2024-02-20T12:54:42.421 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:42.418+0000 7f7948b98700 1 --2- 172.21.15.71:0/1979379804 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f79300310e0 0x7f79300335a0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:54:42.421 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:42.418+0000 7f7948b98700 1 --2- 172.21.15.71:0/1979379804 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f79440f5480 0x7f79441067c0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:54:42.421 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:42.418+0000 7f7948b98700 1 -- 172.21.15.71:0/1979379804 >> 172.21.15.71:0/1979379804 conn(0x7f79440f0be0 msgr2=0x7f794405fee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:54:42.421 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:42.419+0000 7f7948b98700 1 -- 172.21.15.71:0/1979379804 shutdown_connections 2024-02-20T12:54:42.421 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:42.419+0000 7f7948b98700 1 -- 172.21.15.71:0/1979379804 wait complete. 2024-02-20T12:54:42.421 INFO:teuthology.orchestra.run.smithi071.stderr:dumped monmap epoch 1 2024-02-20T12:54:43.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:54:43 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:54:41.738873+0000 mgr.smithi064.vriyor (mgr.14156) 116 : cluster [DBG] pgmap v108: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:54:43.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:54:43 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:54:42.418789+0000 mon.smithi064 (mon.0) 164 : audit [DBG] from='client.? 172.21.15.71:0/1979379804' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-20T12:54:44.066 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-20T12:54:44.066 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/daemon-base:latest-pacific shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 680ee126-cfee-11ee-95bb-87774f69a715 -- ceph mon dump -f json 2024-02-20T12:54:45.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:54:45 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:54:43.739117+0000 mgr.smithi064.vriyor (mgr.14156) 117 : cluster [DBG] pgmap v109: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:54:45.915 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:45.903+0000 7f7d5f4e1700 1 -- 172.21.15.71:0/4165498361 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7d580f6590 msgr2=0x7f7d580f69b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:54:45.915 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:45.903+0000 7f7d5f4e1700 1 --2- 172.21.15.71:0/4165498361 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7d580f6590 0x7f7d580f69b0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 rx=0x7f7d48004760 tx=0x7f7d48009ad0).stop 2024-02-20T12:54:45.915 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:45.903+0000 7f7d5f4e1700 1 -- 172.21.15.71:0/4165498361 shutdown_connections 2024-02-20T12:54:45.915 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:45.903+0000 7f7d5f4e1700 1 --2- 172.21.15.71:0/4165498361 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7d580f6590 0x7f7d580f69b0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:54:45.916 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:45.903+0000 7f7d5f4e1700 1 -- 172.21.15.71:0/4165498361 >> 172.21.15.71:0/4165498361 conn(0x7f7d580f1cd0 msgr2=0x7f7d580f4110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:54:45.916 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:45.903+0000 7f7d5f4e1700 1 -- 172.21.15.71:0/4165498361 shutdown_connections 2024-02-20T12:54:45.916 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:45.903+0000 7f7d5f4e1700 1 -- 172.21.15.71:0/4165498361 wait complete. 2024-02-20T12:54:45.916 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:45.904+0000 7f7d5f4e1700 1 Processor -- start 2024-02-20T12:54:45.916 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:45.904+0000 7f7d5f4e1700 1 -- start start 2024-02-20T12:54:45.916 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:45.904+0000 7f7d5f4e1700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7d580f6590 0x7f7d581067c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:54:45.916 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:45.904+0000 7f7d5f4e1700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d58106cb0 con 0x7f7d580f6590 2024-02-20T12:54:45.916 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:45.904+0000 7f7d5d27d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7d580f6590 0x7f7d581067c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:54:45.916 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:45.904+0000 7f7d5d27d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7d580f6590 0x7f7d581067c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.71:44902/0 (socket says 172.21.15.71:44902) 2024-02-20T12:54:45.916 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:45.904+0000 7f7d5d27d700 1 -- 172.21.15.71:0/1835302426 learned_addr learned my addr 172.21.15.71:0/1835302426 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-02-20T12:54:45.916 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:45.905+0000 7f7d5d27d700 1 -- 172.21.15.71:0/1835302426 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d48009d70 con 0x7f7d580f6590 2024-02-20T12:54:45.916 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:45.905+0000 7f7d5d27d700 1 --2- 172.21.15.71:0/1835302426 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7d580f6590 0x7f7d581067c0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 rx=0x7f7d48004170 tx=0x7f7d48009ad0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:54:45.917 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:45.905+0000 7f7d4dffb700 1 -- 172.21.15.71:0/1835302426 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7d4800e040 con 0x7f7d580f6590 2024-02-20T12:54:45.917 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:45.905+0000 7f7d4dffb700 1 -- 172.21.15.71:0/1835302426 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(25 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f7d48008ef0 con 0x7f7d580f6590 2024-02-20T12:54:45.917 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:45.905+0000 7f7d5f4e1700 1 -- 172.21.15.71:0/1835302426 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d58106eb0 con 0x7f7d580f6590 2024-02-20T12:54:45.917 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:45.905+0000 7f7d4dffb700 1 -- 172.21.15.71:0/1835302426 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7d48014c90 con 0x7f7d580f6590 2024-02-20T12:54:45.917 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:45.905+0000 7f7d5f4e1700 1 -- 172.21.15.71:0/1835302426 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d58107340 con 0x7f7d580f6590 2024-02-20T12:54:45.917 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:45.906+0000 7f7d4dffb700 1 -- 172.21.15.71:0/1835302426 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 37151+0+0 (secure 0 0 0) 0x7f7d48014420 con 0x7f7d580f6590 2024-02-20T12:54:45.917 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:45.906+0000 7f7d5f4e1700 1 -- 172.21.15.71:0/1835302426 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d58045bb0 con 0x7f7d580f6590 2024-02-20T12:54:45.917 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:45.906+0000 7f7d4dffb700 1 --2- 172.21.15.71:0/1835302426 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f7d44031090 0x7f7d44033550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:54:45.917 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:45.906+0000 7f7d4dffb700 1 -- 172.21.15.71:0/1835302426 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1065+0+0 (secure 0 0 0) 0x7f7d48042320 con 0x7f7d580f6590 2024-02-20T12:54:45.917 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:45.906+0000 7f7d5ca7c700 1 --2- 172.21.15.71:0/1835302426 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f7d44031090 0x7f7d44033550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:54:45.917 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:45.907+0000 7f7d5ca7c700 1 --2- 172.21.15.71:0/1835302426 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f7d44031090 0x7f7d44033550 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 rx=0x7f7d540096a0 tx=0x7f7d54006b40).ready entity=mgr.14156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:54:45.917 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:45.911+0000 7f7d4dffb700 1 -- 172.21.15.71:0/1835302426 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+153237 (secure 0 0 0) 0x7f7d48026300 con 0x7f7d580f6590 2024-02-20T12:54:46.120 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:46.118+0000 7f7d5f4e1700 1 -- 172.21.15.71:0/1835302426 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f7d58108190 con 0x7f7d580f6590 2024-02-20T12:54:46.120 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:46.119+0000 7f7d4dffb700 1 -- 172.21.15.71:0/1835302426 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7f7d48012b20 con 0x7f7d580f6590 2024-02-20T12:54:46.121 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-20T12:54:46.121 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":1,"fsid":"680ee126-cfee-11ee-95bb-87774f69a715","modified":"2024-02-20T12:49:44.699059Z","created":"2024-02-20T12:49:44.699059Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi064","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:3300","nonce":0},{"type":"v1","addr":"172.21.15.64:6789","nonce":0}]},"addr":"172.21.15.64:6789/0","public_addr":"172.21.15.64:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-20T12:54:46.122 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:46.121+0000 7f7d5f4e1700 1 -- 172.21.15.71:0/1835302426 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f7d44031090 msgr2=0x7f7d44033550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:54:46.123 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:46.121+0000 7f7d5f4e1700 1 --2- 172.21.15.71:0/1835302426 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f7d44031090 0x7f7d44033550 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 rx=0x7f7d540096a0 tx=0x7f7d54006b40).stop 2024-02-20T12:54:46.123 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:46.121+0000 7f7d5f4e1700 1 -- 172.21.15.71:0/1835302426 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7d580f6590 msgr2=0x7f7d581067c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:54:46.123 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:46.121+0000 7f7d5f4e1700 1 --2- 172.21.15.71:0/1835302426 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7d580f6590 0x7f7d581067c0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 rx=0x7f7d48004170 tx=0x7f7d48009ad0).stop 2024-02-20T12:54:46.123 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:46.121+0000 7f7d5f4e1700 1 -- 172.21.15.71:0/1835302426 shutdown_connections 2024-02-20T12:54:46.123 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:46.121+0000 7f7d5f4e1700 1 --2- 172.21.15.71:0/1835302426 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f7d44031090 0x7f7d44033550 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:54:46.123 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:46.121+0000 7f7d5f4e1700 1 --2- 172.21.15.71:0/1835302426 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7d580f6590 0x7f7d581067c0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:54:46.123 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:46.121+0000 7f7d5f4e1700 1 -- 172.21.15.71:0/1835302426 >> 172.21.15.71:0/1835302426 conn(0x7f7d580f1cd0 msgr2=0x7f7d580f27c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:54:46.123 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:46.121+0000 7f7d5f4e1700 1 -- 172.21.15.71:0/1835302426 shutdown_connections 2024-02-20T12:54:46.123 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:46.121+0000 7f7d5f4e1700 1 -- 172.21.15.71:0/1835302426 wait complete. 2024-02-20T12:54:46.123 INFO:teuthology.orchestra.run.smithi071.stderr:dumped monmap epoch 1 2024-02-20T12:54:46.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:54:46 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:54:46.120619+0000 mon.smithi064 (mon.0) 165 : audit [DBG] from='client.? 172.21.15.71:0/1835302426' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-20T12:54:47.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:54:47 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:54:45.739466+0000 mgr.smithi064.vriyor (mgr.14156) 118 : cluster [DBG] pgmap v110: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:54:47.855 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-20T12:54:47.856 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/daemon-base:latest-pacific shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 680ee126-cfee-11ee-95bb-87774f69a715 -- ceph mon dump -f json 2024-02-20T12:54:49.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:54:49 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:54:47.739860+0000 mgr.smithi064.vriyor (mgr.14156) 119 : cluster [DBG] pgmap v111: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:54:49.850 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:49.847+0000 7f0e51630700 1 -- 172.21.15.71:0/4135152869 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0e4c060360 msgr2=0x7f0e4c0f8fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:54:49.851 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:49.847+0000 7f0e51630700 1 --2- 172.21.15.71:0/4135152869 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0e4c060360 0x7f0e4c0f8fb0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 rx=0x7f0e3c009fb0 tx=0x7f0e3c009b30).stop 2024-02-20T12:54:49.851 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:49.847+0000 7f0e51630700 1 -- 172.21.15.71:0/4135152869 shutdown_connections 2024-02-20T12:54:49.851 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:49.847+0000 7f0e51630700 1 --2- 172.21.15.71:0/4135152869 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0e4c060360 0x7f0e4c0f8fb0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:54:49.851 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:49.847+0000 7f0e51630700 1 -- 172.21.15.71:0/4135152869 >> 172.21.15.71:0/4135152869 conn(0x7f0e4c0f0a30 msgr2=0x7f0e4c0f2e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:54:49.851 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:49.847+0000 7f0e51630700 1 -- 172.21.15.71:0/4135152869 shutdown_connections 2024-02-20T12:54:49.851 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:49.847+0000 7f0e51630700 1 -- 172.21.15.71:0/4135152869 wait complete. 2024-02-20T12:54:49.851 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:49.848+0000 7f0e51630700 1 Processor -- start 2024-02-20T12:54:49.851 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:49.848+0000 7f0e51630700 1 -- start start 2024-02-20T12:54:49.851 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:49.848+0000 7f0e51630700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0e4c060360 0x7f0e4c0f6a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:54:49.851 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:49.848+0000 7f0e51630700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e4c0f4eb0 con 0x7f0e4c060360 2024-02-20T12:54:49.851 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:49.848+0000 7f0e4bfff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0e4c060360 0x7f0e4c0f6a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:54:49.852 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:49.848+0000 7f0e4bfff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0e4c060360 0x7f0e4c0f6a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.71:39496/0 (socket says 172.21.15.71:39496) 2024-02-20T12:54:49.852 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:49.848+0000 7f0e4bfff700 1 -- 172.21.15.71:0/3997795189 learned_addr learned my addr 172.21.15.71:0/3997795189 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-02-20T12:54:49.852 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:49.848+0000 7f0e4bfff700 1 -- 172.21.15.71:0/3997795189 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e3c005040 con 0x7f0e4c060360 2024-02-20T12:54:49.852 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:49.849+0000 7f0e4bfff700 1 --2- 172.21.15.71:0/3997795189 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0e4c060360 0x7f0e4c0f6a30 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 rx=0x7f0e3c009f80 tx=0x7f0e3c00b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:54:49.852 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:49.849+0000 7f0e48ff9700 1 -- 172.21.15.71:0/3997795189 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0e3c009320 con 0x7f0e4c060360 2024-02-20T12:54:49.852 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:49.849+0000 7f0e51630700 1 -- 172.21.15.71:0/3997795189 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e4c0f5050 con 0x7f0e4c060360 2024-02-20T12:54:49.852 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:49.849+0000 7f0e48ff9700 1 -- 172.21.15.71:0/3997795189 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(25 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f0e3c013680 con 0x7f0e4c060360 2024-02-20T12:54:49.852 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:49.849+0000 7f0e48ff9700 1 -- 172.21.15.71:0/3997795189 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0e3c01eda0 con 0x7f0e4c060360 2024-02-20T12:54:49.852 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:49.849+0000 7f0e51630700 1 -- 172.21.15.71:0/3997795189 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e4c0f54e0 con 0x7f0e4c060360 2024-02-20T12:54:49.853 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:49.850+0000 7f0e48ff9700 1 -- 172.21.15.71:0/3997795189 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 37151+0+0 (secure 0 0 0) 0x7f0e3c01e420 con 0x7f0e4c060360 2024-02-20T12:54:49.853 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:49.850+0000 7f0e51630700 1 -- 172.21.15.71:0/3997795189 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e4c0591e0 con 0x7f0e4c060360 2024-02-20T12:54:49.853 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:49.850+0000 7f0e48ff9700 1 --2- 172.21.15.71:0/3997795189 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f0e34031090 0x7f0e34033550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:54:49.853 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:49.850+0000 7f0e48ff9700 1 -- 172.21.15.71:0/3997795189 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1065+0+0 (secure 0 0 0) 0x7f0e3c042390 con 0x7f0e4c060360 2024-02-20T12:54:49.853 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:49.850+0000 7f0e4b7fe700 1 --2- 172.21.15.71:0/3997795189 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f0e34031090 0x7f0e34033550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:54:49.853 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:49.851+0000 7f0e4b7fe700 1 --2- 172.21.15.71:0/3997795189 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f0e34031090 0x7f0e34033550 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 rx=0x7f0e400096a0 tx=0x7f0e40006b40).ready entity=mgr.14156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:54:49.856 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:49.855+0000 7f0e48ff9700 1 -- 172.21.15.71:0/3997795189 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+153237 (secure 0 0 0) 0x7f0e3c01d380 con 0x7f0e4c060360 2024-02-20T12:54:50.063 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:50.062+0000 7f0e51630700 1 -- 172.21.15.71:0/3997795189 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f0e4c0fbeb0 con 0x7f0e4c060360 2024-02-20T12:54:50.064 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:50.063+0000 7f0e48ff9700 1 -- 172.21.15.71:0/3997795189 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7f0e3c004490 con 0x7f0e4c060360 2024-02-20T12:54:50.064 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-20T12:54:50.064 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":1,"fsid":"680ee126-cfee-11ee-95bb-87774f69a715","modified":"2024-02-20T12:49:44.699059Z","created":"2024-02-20T12:49:44.699059Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi064","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:3300","nonce":0},{"type":"v1","addr":"172.21.15.64:6789","nonce":0}]},"addr":"172.21.15.64:6789/0","public_addr":"172.21.15.64:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-20T12:54:50.066 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:50.064+0000 7f0e51630700 1 -- 172.21.15.71:0/3997795189 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f0e34031090 msgr2=0x7f0e34033550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:54:50.066 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:50.064+0000 7f0e51630700 1 --2- 172.21.15.71:0/3997795189 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f0e34031090 0x7f0e34033550 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 rx=0x7f0e400096a0 tx=0x7f0e40006b40).stop 2024-02-20T12:54:50.066 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:50.064+0000 7f0e51630700 1 -- 172.21.15.71:0/3997795189 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0e4c060360 msgr2=0x7f0e4c0f6a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:54:50.066 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:50.064+0000 7f0e51630700 1 --2- 172.21.15.71:0/3997795189 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0e4c060360 0x7f0e4c0f6a30 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 rx=0x7f0e3c009f80 tx=0x7f0e3c00b040).stop 2024-02-20T12:54:50.066 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:50.064+0000 7f0e51630700 1 -- 172.21.15.71:0/3997795189 shutdown_connections 2024-02-20T12:54:50.066 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:50.064+0000 7f0e51630700 1 --2- 172.21.15.71:0/3997795189 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f0e34031090 0x7f0e34033550 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:54:50.066 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:50.064+0000 7f0e51630700 1 --2- 172.21.15.71:0/3997795189 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0e4c060360 0x7f0e4c0f6a30 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:54:50.067 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:50.065+0000 7f0e51630700 1 -- 172.21.15.71:0/3997795189 >> 172.21.15.71:0/3997795189 conn(0x7f0e4c0f0a30 msgr2=0x7f0e4c0f14e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:54:50.067 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:50.065+0000 7f0e51630700 1 -- 172.21.15.71:0/3997795189 shutdown_connections 2024-02-20T12:54:50.067 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:50.065+0000 7f0e51630700 1 -- 172.21.15.71:0/3997795189 wait complete. 2024-02-20T12:54:50.067 INFO:teuthology.orchestra.run.smithi071.stderr:dumped monmap epoch 1 2024-02-20T12:54:50.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:54:50 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:54:50.064481+0000 mon.smithi064 (mon.0) 166 : audit [DBG] from='client.? 172.21.15.71:0/3997795189' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-20T12:54:51.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:54:51 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:54:49.740209+0000 mgr.smithi064.vriyor (mgr.14156) 120 : cluster [DBG] pgmap v112: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:54:51.708 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-20T12:54:51.708 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/daemon-base:latest-pacific shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 680ee126-cfee-11ee-95bb-87774f69a715 -- ceph mon dump -f json 2024-02-20T12:54:53.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:54:53 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:54:51.740554+0000 mgr.smithi064.vriyor (mgr.14156) 121 : cluster [DBG] pgmap v113: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:54:53.643 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:53.614+0000 7fb592e98700 1 -- 172.21.15.71:0/1119351808 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb58c0f6590 msgr2=0x7fb58c0f69b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:54:53.643 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:53.614+0000 7fb592e98700 1 --2- 172.21.15.71:0/1119351808 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb58c0f6590 0x7fb58c0f69b0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 rx=0x7fb580004760 tx=0x7fb580009ad0).stop 2024-02-20T12:54:53.643 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:53.614+0000 7fb592e98700 1 -- 172.21.15.71:0/1119351808 shutdown_connections 2024-02-20T12:54:53.643 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:53.614+0000 7fb592e98700 1 --2- 172.21.15.71:0/1119351808 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb58c0f6590 0x7fb58c0f69b0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:54:53.643 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:53.614+0000 7fb592e98700 1 -- 172.21.15.71:0/1119351808 >> 172.21.15.71:0/1119351808 conn(0x7fb58c0f1cd0 msgr2=0x7fb58c0f4110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:54:53.643 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:53.614+0000 7fb592e98700 1 -- 172.21.15.71:0/1119351808 shutdown_connections 2024-02-20T12:54:53.643 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:53.614+0000 7fb592e98700 1 -- 172.21.15.71:0/1119351808 wait complete. 2024-02-20T12:54:53.643 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:53.615+0000 7fb592e98700 1 Processor -- start 2024-02-20T12:54:53.644 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:53.615+0000 7fb592e98700 1 -- start start 2024-02-20T12:54:53.644 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:53.615+0000 7fb592e98700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb58c0f6590 0x7fb58c1067c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:54:53.644 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:53.615+0000 7fb592e98700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb58c106cb0 con 0x7fb58c0f6590 2024-02-20T12:54:53.644 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:53.615+0000 7fb590c34700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb58c0f6590 0x7fb58c1067c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:54:53.644 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:53.615+0000 7fb590c34700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb58c0f6590 0x7fb58c1067c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.71:39510/0 (socket says 172.21.15.71:39510) 2024-02-20T12:54:53.644 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:53.615+0000 7fb590c34700 1 -- 172.21.15.71:0/2050757685 learned_addr learned my addr 172.21.15.71:0/2050757685 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-02-20T12:54:53.644 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:53.616+0000 7fb590c34700 1 -- 172.21.15.71:0/2050757685 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb580009d70 con 0x7fb58c0f6590 2024-02-20T12:54:53.644 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:53.616+0000 7fb590c34700 1 --2- 172.21.15.71:0/2050757685 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb58c0f6590 0x7fb58c1067c0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 rx=0x7fb580004170 tx=0x7fb580009ad0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:54:53.644 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:53.616+0000 7fb5897fa700 1 -- 172.21.15.71:0/2050757685 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb58000e040 con 0x7fb58c0f6590 2024-02-20T12:54:53.644 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:53.616+0000 7fb592e98700 1 -- 172.21.15.71:0/2050757685 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb58c106eb0 con 0x7fb58c0f6590 2024-02-20T12:54:53.644 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:53.616+0000 7fb5897fa700 1 -- 172.21.15.71:0/2050757685 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(25 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7fb580008ef0 con 0x7fb58c0f6590 2024-02-20T12:54:53.644 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:53.616+0000 7fb5897fa700 1 -- 172.21.15.71:0/2050757685 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb580014c90 con 0x7fb58c0f6590 2024-02-20T12:54:53.645 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:53.616+0000 7fb592e98700 1 -- 172.21.15.71:0/2050757685 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb58c107340 con 0x7fb58c0f6590 2024-02-20T12:54:53.645 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:53.617+0000 7fb5897fa700 1 -- 172.21.15.71:0/2050757685 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 37151+0+0 (secure 0 0 0) 0x7fb580014420 con 0x7fb58c0f6590 2024-02-20T12:54:53.645 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:53.617+0000 7fb5897fa700 1 --2- 172.21.15.71:0/2050757685 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fb57c031090 0x7fb57c033550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:54:53.645 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:53.617+0000 7fb5897fa700 1 -- 172.21.15.71:0/2050757685 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1065+0+0 (secure 0 0 0) 0x7fb580042300 con 0x7fb58c0f6590 2024-02-20T12:54:53.645 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:53.617+0000 7fb592e98700 1 -- 172.21.15.71:0/2050757685 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb58c045bb0 con 0x7fb58c0f6590 2024-02-20T12:54:53.645 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:53.617+0000 7fb58bfff700 1 --2- 172.21.15.71:0/2050757685 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fb57c031090 0x7fb57c033550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:54:53.645 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:53.618+0000 7fb58bfff700 1 --2- 172.21.15.71:0/2050757685 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fb57c031090 0x7fb57c033550 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 rx=0x7fb5780096a0 tx=0x7fb578006b40).ready entity=mgr.14156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:54:53.645 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:53.622+0000 7fb5897fa700 1 -- 172.21.15.71:0/2050757685 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+153237 (secure 0 0 0) 0x7fb580012d00 con 0x7fb58c0f6590 2024-02-20T12:54:53.832 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:53.831+0000 7fb592e98700 1 -- 172.21.15.71:0/2050757685 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fb58c0fc170 con 0x7fb58c0f6590 2024-02-20T12:54:53.833 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:53.831+0000 7fb5897fa700 1 -- 172.21.15.71:0/2050757685 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7fb580012b20 con 0x7fb58c0f6590 2024-02-20T12:54:53.833 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-20T12:54:53.833 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":1,"fsid":"680ee126-cfee-11ee-95bb-87774f69a715","modified":"2024-02-20T12:49:44.699059Z","created":"2024-02-20T12:49:44.699059Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi064","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:3300","nonce":0},{"type":"v1","addr":"172.21.15.64:6789","nonce":0}]},"addr":"172.21.15.64:6789/0","public_addr":"172.21.15.64:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-20T12:54:53.834 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:53.833+0000 7fb592e98700 1 -- 172.21.15.71:0/2050757685 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fb57c031090 msgr2=0x7fb57c033550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:54:53.835 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:53.833+0000 7fb592e98700 1 --2- 172.21.15.71:0/2050757685 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fb57c031090 0x7fb57c033550 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 rx=0x7fb5780096a0 tx=0x7fb578006b40).stop 2024-02-20T12:54:53.835 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:53.833+0000 7fb592e98700 1 -- 172.21.15.71:0/2050757685 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb58c0f6590 msgr2=0x7fb58c1067c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:54:53.835 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:53.833+0000 7fb592e98700 1 --2- 172.21.15.71:0/2050757685 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb58c0f6590 0x7fb58c1067c0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 rx=0x7fb580004170 tx=0x7fb580009ad0).stop 2024-02-20T12:54:53.835 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:53.833+0000 7fb592e98700 1 -- 172.21.15.71:0/2050757685 shutdown_connections 2024-02-20T12:54:53.835 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:53.833+0000 7fb592e98700 1 --2- 172.21.15.71:0/2050757685 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fb57c031090 0x7fb57c033550 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:54:53.835 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:53.833+0000 7fb592e98700 1 --2- 172.21.15.71:0/2050757685 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb58c0f6590 0x7fb58c1067c0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:54:53.835 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:53.833+0000 7fb592e98700 1 -- 172.21.15.71:0/2050757685 >> 172.21.15.71:0/2050757685 conn(0x7fb58c0f1cd0 msgr2=0x7fb58c0f27c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:54:53.835 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:53.833+0000 7fb592e98700 1 -- 172.21.15.71:0/2050757685 shutdown_connections 2024-02-20T12:54:53.835 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:53.833+0000 7fb592e98700 1 -- 172.21.15.71:0/2050757685 wait complete. 2024-02-20T12:54:53.835 INFO:teuthology.orchestra.run.smithi071.stderr:dumped monmap epoch 1 2024-02-20T12:54:54.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:54:54 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:54:53.833278+0000 mon.smithi064 (mon.0) 167 : audit [DBG] from='client.? 172.21.15.71:0/2050757685' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-20T12:54:54.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:54:54 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:54:53.846252+0000 mon.smithi064 (mon.0) 168 : audit [INF] from='mgr.14156 172.21.15.64:0/3284974785' entity='mgr.smithi064.vriyor' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi064.vriyor/mirror_snapshot_schedule"}]: dispatch 2024-02-20T12:54:54.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:54:54 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:54:53.847041+0000 mon.smithi064 (mon.0) 169 : audit [INF] from='mgr.14156 172.21.15.64:0/3284974785' entity='mgr.smithi064.vriyor' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi064.vriyor/trash_purge_schedule"}]: dispatch 2024-02-20T12:54:55.561 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-20T12:54:55.562 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/daemon-base:latest-pacific shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 680ee126-cfee-11ee-95bb-87774f69a715 -- ceph mon dump -f json 2024-02-20T12:54:55.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:54:55 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:54:53.740819+0000 mgr.smithi064.vriyor (mgr.14156) 122 : cluster [DBG] pgmap v114: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:54:57.505 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:57.501+0000 7f9f93538700 1 -- 172.21.15.71:0/2457504714 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9f8c0f5480 msgr2=0x7f9f8c0f58a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:54:57.505 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:57.501+0000 7f9f93538700 1 --2- 172.21.15.71:0/2457504714 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9f8c0f5480 0x7f9f8c0f58a0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 rx=0x7f9f7c009fb0 tx=0x7f9f7c009b30).stop 2024-02-20T12:54:57.505 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:57.502+0000 7f9f93538700 1 -- 172.21.15.71:0/2457504714 shutdown_connections 2024-02-20T12:54:57.505 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:57.502+0000 7f9f93538700 1 --2- 172.21.15.71:0/2457504714 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9f8c0f5480 0x7f9f8c0f58a0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:54:57.505 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:57.502+0000 7f9f93538700 1 -- 172.21.15.71:0/2457504714 >> 172.21.15.71:0/2457504714 conn(0x7f9f8c0f0be0 msgr2=0x7f9f8c0f3000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:54:57.505 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:57.502+0000 7f9f93538700 1 -- 172.21.15.71:0/2457504714 shutdown_connections 2024-02-20T12:54:57.505 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:57.502+0000 7f9f93538700 1 -- 172.21.15.71:0/2457504714 wait complete. 2024-02-20T12:54:57.505 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:57.502+0000 7f9f93538700 1 Processor -- start 2024-02-20T12:54:57.505 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:57.502+0000 7f9f93538700 1 -- start start 2024-02-20T12:54:57.506 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:57.502+0000 7f9f93538700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9f8c0f5480 0x7f9f8c1056f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:54:57.506 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:57.502+0000 7f9f93538700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f8c105be0 con 0x7f9f8c0f5480 2024-02-20T12:54:57.506 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:57.503+0000 7f9f912d4700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9f8c0f5480 0x7f9f8c1056f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:54:57.506 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:57.503+0000 7f9f912d4700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9f8c0f5480 0x7f9f8c1056f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.71:45528/0 (socket says 172.21.15.71:45528) 2024-02-20T12:54:57.506 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:57.503+0000 7f9f912d4700 1 -- 172.21.15.71:0/3616686842 learned_addr learned my addr 172.21.15.71:0/3616686842 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-02-20T12:54:57.506 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:57.503+0000 7f9f912d4700 1 -- 172.21.15.71:0/3616686842 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f7c005040 con 0x7f9f8c0f5480 2024-02-20T12:54:57.506 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:57.503+0000 7f9f912d4700 1 --2- 172.21.15.71:0/3616686842 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9f8c0f5480 0x7f9f8c1056f0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 rx=0x7f9f7c0089f0 tx=0x7f9f7c00b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:54:57.506 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:57.503+0000 7f9f81ffb700 1 -- 172.21.15.71:0/3616686842 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9f7c009320 con 0x7f9f8c0f5480 2024-02-20T12:54:57.506 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:57.503+0000 7f9f81ffb700 1 -- 172.21.15.71:0/3616686842 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(25 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f9f7c013ec0 con 0x7f9f8c0f5480 2024-02-20T12:54:57.506 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:57.503+0000 7f9f93538700 1 -- 172.21.15.71:0/3616686842 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f8c105de0 con 0x7f9f8c0f5480 2024-02-20T12:54:57.506 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:57.503+0000 7f9f81ffb700 1 -- 172.21.15.71:0/3616686842 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9f7c014d30 con 0x7f9f8c0f5480 2024-02-20T12:54:57.506 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:57.503+0000 7f9f93538700 1 -- 172.21.15.71:0/3616686842 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f8c106270 con 0x7f9f8c0f5480 2024-02-20T12:54:57.507 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:57.504+0000 7f9f81ffb700 1 -- 172.21.15.71:0/3616686842 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 37151+0+0 (secure 0 0 0) 0x7f9f7c014420 con 0x7f9f8c0f5480 2024-02-20T12:54:57.507 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:57.504+0000 7f9f81ffb700 1 --2- 172.21.15.71:0/3616686842 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f9f78031090 0x7f9f78033550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:54:57.507 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:57.504+0000 7f9f93538700 1 -- 172.21.15.71:0/3616686842 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f8c045bb0 con 0x7f9f8c0f5480 2024-02-20T12:54:57.507 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:57.505+0000 7f9f81ffb700 1 -- 172.21.15.71:0/3616686842 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1065+0+0 (secure 0 0 0) 0x7f9f7c042400 con 0x7f9f8c0f5480 2024-02-20T12:54:57.507 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:57.505+0000 7f9f90ad3700 1 --2- 172.21.15.71:0/3616686842 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f9f78031090 0x7f9f78033550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:54:57.508 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:57.505+0000 7f9f90ad3700 1 --2- 172.21.15.71:0/3616686842 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f9f78031090 0x7f9f78033550 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 rx=0x7f9f880096a0 tx=0x7f9f88006b40).ready entity=mgr.14156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:54:57.511 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:57.510+0000 7f9f81ffb700 1 -- 172.21.15.71:0/3616686842 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+153237 (secure 0 0 0) 0x7f9f7c014750 con 0x7f9f8c0f5480 2024-02-20T12:54:57.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:54:57 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:54:55.741178+0000 mgr.smithi064.vriyor (mgr.14156) 123 : cluster [DBG] pgmap v115: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:54:57.716 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:57.714+0000 7f9f93538700 1 -- 172.21.15.71:0/3616686842 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f9f8c107070 con 0x7f9f8c0f5480 2024-02-20T12:54:57.716 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:57.715+0000 7f9f81ffb700 1 -- 172.21.15.71:0/3616686842 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7f9f7c01d380 con 0x7f9f8c0f5480 2024-02-20T12:54:57.716 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-20T12:54:57.717 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":1,"fsid":"680ee126-cfee-11ee-95bb-87774f69a715","modified":"2024-02-20T12:49:44.699059Z","created":"2024-02-20T12:49:44.699059Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi064","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:3300","nonce":0},{"type":"v1","addr":"172.21.15.64:6789","nonce":0}]},"addr":"172.21.15.64:6789/0","public_addr":"172.21.15.64:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-20T12:54:57.718 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:57.717+0000 7f9f93538700 1 -- 172.21.15.71:0/3616686842 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f9f78031090 msgr2=0x7f9f78033550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:54:57.718 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:57.717+0000 7f9f93538700 1 --2- 172.21.15.71:0/3616686842 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f9f78031090 0x7f9f78033550 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 rx=0x7f9f880096a0 tx=0x7f9f88006b40).stop 2024-02-20T12:54:57.718 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:57.717+0000 7f9f93538700 1 -- 172.21.15.71:0/3616686842 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9f8c0f5480 msgr2=0x7f9f8c1056f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:54:57.718 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:57.717+0000 7f9f93538700 1 --2- 172.21.15.71:0/3616686842 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9f8c0f5480 0x7f9f8c1056f0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 rx=0x7f9f7c0089f0 tx=0x7f9f7c00b040).stop 2024-02-20T12:54:57.718 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:57.717+0000 7f9f93538700 1 -- 172.21.15.71:0/3616686842 shutdown_connections 2024-02-20T12:54:57.718 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:57.717+0000 7f9f93538700 1 --2- 172.21.15.71:0/3616686842 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f9f78031090 0x7f9f78033550 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:54:57.718 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:57.717+0000 7f9f93538700 1 --2- 172.21.15.71:0/3616686842 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9f8c0f5480 0x7f9f8c1056f0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:54:57.718 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:57.717+0000 7f9f93538700 1 -- 172.21.15.71:0/3616686842 >> 172.21.15.71:0/3616686842 conn(0x7f9f8c0f0be0 msgr2=0x7f9f8c0f16d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:54:57.719 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:57.717+0000 7f9f93538700 1 -- 172.21.15.71:0/3616686842 shutdown_connections 2024-02-20T12:54:57.719 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:54:57.717+0000 7f9f93538700 1 -- 172.21.15.71:0/3616686842 wait complete. 2024-02-20T12:54:57.719 INFO:teuthology.orchestra.run.smithi071.stderr:dumped monmap epoch 1 2024-02-20T12:54:58.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:54:58 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:54:57.716725+0000 mon.smithi064 (mon.0) 170 : audit [DBG] from='client.? 172.21.15.71:0/3616686842' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-20T12:54:59.357 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-20T12:54:59.358 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/daemon-base:latest-pacific shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 680ee126-cfee-11ee-95bb-87774f69a715 -- ceph mon dump -f json 2024-02-20T12:54:59.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:54:59 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:54:57.741528+0000 mgr.smithi064.vriyor (mgr.14156) 124 : cluster [DBG] pgmap v116: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:55:01.313 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:01.294+0000 7f105b9c7700 1 -- 172.21.15.71:0/3865528989 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f10540f5480 msgr2=0x7f10540f58a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:55:01.313 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:01.294+0000 7f105b9c7700 1 --2- 172.21.15.71:0/3865528989 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f10540f5480 0x7f10540f58a0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 rx=0x7f1044009fb0 tx=0x7f1044009b30).stop 2024-02-20T12:55:01.313 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:01.294+0000 7f105b9c7700 1 -- 172.21.15.71:0/3865528989 shutdown_connections 2024-02-20T12:55:01.313 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:01.294+0000 7f105b9c7700 1 --2- 172.21.15.71:0/3865528989 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f10540f5480 0x7f10540f58a0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:55:01.313 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:01.294+0000 7f105b9c7700 1 -- 172.21.15.71:0/3865528989 >> 172.21.15.71:0/3865528989 conn(0x7f10540f0be0 msgr2=0x7f10540f3000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:55:01.313 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:01.294+0000 7f105b9c7700 1 -- 172.21.15.71:0/3865528989 shutdown_connections 2024-02-20T12:55:01.314 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:01.294+0000 7f105b9c7700 1 -- 172.21.15.71:0/3865528989 wait complete. 2024-02-20T12:55:01.314 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:01.295+0000 7f105b9c7700 1 Processor -- start 2024-02-20T12:55:01.314 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:01.295+0000 7f105b9c7700 1 -- start start 2024-02-20T12:55:01.314 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:01.295+0000 7f105b9c7700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f10540f5480 0x7f10541094b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:55:01.314 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:01.295+0000 7f105b9c7700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1054060360 con 0x7f10540f5480 2024-02-20T12:55:01.314 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:01.295+0000 7f1059763700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f10540f5480 0x7f10541094b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:55:01.314 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:01.295+0000 7f1059763700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f10540f5480 0x7f10541094b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.71:45554/0 (socket says 172.21.15.71:45554) 2024-02-20T12:55:01.315 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:01.295+0000 7f1059763700 1 -- 172.21.15.71:0/1977338714 learned_addr learned my addr 172.21.15.71:0/1977338714 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-02-20T12:55:01.315 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:01.296+0000 7f1059763700 1 -- 172.21.15.71:0/1977338714 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1044005040 con 0x7f10540f5480 2024-02-20T12:55:01.315 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:01.296+0000 7f1059763700 1 --2- 172.21.15.71:0/1977338714 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f10540f5480 0x7f10541094b0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 rx=0x7f10440089f0 tx=0x7f104400b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:55:01.315 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:01.296+0000 7f1049ffb700 1 -- 172.21.15.71:0/1977338714 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1044009320 con 0x7f10540f5480 2024-02-20T12:55:01.315 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:01.296+0000 7f105b9c7700 1 -- 172.21.15.71:0/1977338714 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1054060560 con 0x7f10540f5480 2024-02-20T12:55:01.316 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:01.296+0000 7f1049ffb700 1 -- 172.21.15.71:0/1977338714 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(25 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f1044013ec0 con 0x7f10540f5480 2024-02-20T12:55:01.316 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:01.296+0000 7f1049ffb700 1 -- 172.21.15.71:0/1977338714 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1044014d30 con 0x7f10540f5480 2024-02-20T12:55:01.316 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:01.296+0000 7f105b9c7700 1 -- 172.21.15.71:0/1977338714 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1054109be0 con 0x7f10540f5480 2024-02-20T12:55:01.316 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:01.297+0000 7f1049ffb700 1 -- 172.21.15.71:0/1977338714 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 37151+0+0 (secure 0 0 0) 0x7f10440095f0 con 0x7f10540f5480 2024-02-20T12:55:01.316 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:01.297+0000 7f105b9c7700 1 -- 172.21.15.71:0/1977338714 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1054045bb0 con 0x7f10540f5480 2024-02-20T12:55:01.316 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:01.297+0000 7f1049ffb700 1 --2- 172.21.15.71:0/1977338714 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f1040031090 0x7f1040033550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:55:01.317 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:01.297+0000 7f1049ffb700 1 -- 172.21.15.71:0/1977338714 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1065+0+0 (secure 0 0 0) 0x7f10440423e0 con 0x7f10540f5480 2024-02-20T12:55:01.317 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:01.297+0000 7f1058f62700 1 --2- 172.21.15.71:0/1977338714 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f1040031090 0x7f1040033550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:55:01.317 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:01.298+0000 7f1058f62700 1 --2- 172.21.15.71:0/1977338714 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f1040031090 0x7f1040033550 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 rx=0x7f10500096a0 tx=0x7f1050006b40).ready entity=mgr.14156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:55:01.317 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:01.302+0000 7f1049ffb700 1 -- 172.21.15.71:0/1977338714 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+153237 (secure 0 0 0) 0x7f104401d7f0 con 0x7f10540f5480 2024-02-20T12:55:01.508 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:01.506+0000 7f105b9c7700 1 -- 172.21.15.71:0/1977338714 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f105410a400 con 0x7f10540f5480 2024-02-20T12:55:01.508 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:01.507+0000 7f1049ffb700 1 -- 172.21.15.71:0/1977338714 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7f104401d7f0 con 0x7f10540f5480 2024-02-20T12:55:01.509 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-20T12:55:01.509 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":1,"fsid":"680ee126-cfee-11ee-95bb-87774f69a715","modified":"2024-02-20T12:49:44.699059Z","created":"2024-02-20T12:49:44.699059Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi064","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:3300","nonce":0},{"type":"v1","addr":"172.21.15.64:6789","nonce":0}]},"addr":"172.21.15.64:6789/0","public_addr":"172.21.15.64:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-20T12:55:01.510 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:01.509+0000 7f105b9c7700 1 -- 172.21.15.71:0/1977338714 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f1040031090 msgr2=0x7f1040033550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:55:01.510 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:01.509+0000 7f105b9c7700 1 --2- 172.21.15.71:0/1977338714 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f1040031090 0x7f1040033550 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 rx=0x7f10500096a0 tx=0x7f1050006b40).stop 2024-02-20T12:55:01.510 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:01.509+0000 7f105b9c7700 1 -- 172.21.15.71:0/1977338714 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f10540f5480 msgr2=0x7f10541094b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:55:01.510 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:01.509+0000 7f105b9c7700 1 --2- 172.21.15.71:0/1977338714 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f10540f5480 0x7f10541094b0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 rx=0x7f10440089f0 tx=0x7f104400b040).stop 2024-02-20T12:55:01.510 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:01.509+0000 7f105b9c7700 1 -- 172.21.15.71:0/1977338714 shutdown_connections 2024-02-20T12:55:01.511 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:01.509+0000 7f105b9c7700 1 --2- 172.21.15.71:0/1977338714 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f1040031090 0x7f1040033550 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:55:01.511 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:01.509+0000 7f105b9c7700 1 --2- 172.21.15.71:0/1977338714 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f10540f5480 0x7f10541094b0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:55:01.511 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:01.509+0000 7f105b9c7700 1 -- 172.21.15.71:0/1977338714 >> 172.21.15.71:0/1977338714 conn(0x7f10540f0be0 msgr2=0x7f10540f16d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:55:01.511 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:01.509+0000 7f105b9c7700 1 -- 172.21.15.71:0/1977338714 shutdown_connections 2024-02-20T12:55:01.511 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:01.509+0000 7f105b9c7700 1 -- 172.21.15.71:0/1977338714 wait complete. 2024-02-20T12:55:01.511 INFO:teuthology.orchestra.run.smithi071.stderr:dumped monmap epoch 1 2024-02-20T12:55:01.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:55:01 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:54:59.741810+0000 mgr.smithi064.vriyor (mgr.14156) 125 : cluster [DBG] pgmap v117: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:55:02.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:55:02 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:55:01.508835+0000 mon.smithi064 (mon.0) 171 : audit [DBG] from='client.? 172.21.15.71:0/1977338714' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-20T12:55:03.243 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-20T12:55:03.243 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/daemon-base:latest-pacific shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 680ee126-cfee-11ee-95bb-87774f69a715 -- ceph mon dump -f json 2024-02-20T12:55:03.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:55:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:55:01.742160+0000 mgr.smithi064.vriyor (mgr.14156) 126 : cluster [DBG] pgmap v118: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:55:05.210 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:05.208+0000 7f2c50afc700 1 -- 172.21.15.71:0/3627739965 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2c4c0f5480 msgr2=0x7f2c4c0f58a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:55:05.210 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:05.208+0000 7f2c50afc700 1 --2- 172.21.15.71:0/3627739965 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2c4c0f5480 0x7f2c4c0f58a0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 rx=0x7f2c3c009fb0 tx=0x7f2c3c009b30).stop 2024-02-20T12:55:05.210 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:05.208+0000 7f2c50afc700 1 -- 172.21.15.71:0/3627739965 shutdown_connections 2024-02-20T12:55:05.210 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:05.208+0000 7f2c50afc700 1 --2- 172.21.15.71:0/3627739965 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2c4c0f5480 0x7f2c4c0f58a0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:55:05.210 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:05.208+0000 7f2c50afc700 1 -- 172.21.15.71:0/3627739965 >> 172.21.15.71:0/3627739965 conn(0x7f2c4c0f0be0 msgr2=0x7f2c4c0f3000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:55:05.210 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:05.208+0000 7f2c50afc700 1 -- 172.21.15.71:0/3627739965 shutdown_connections 2024-02-20T12:55:05.211 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:05.208+0000 7f2c50afc700 1 -- 172.21.15.71:0/3627739965 wait complete. 2024-02-20T12:55:05.211 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:05.208+0000 7f2c50afc700 1 Processor -- start 2024-02-20T12:55:05.211 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:05.209+0000 7f2c50afc700 1 -- start start 2024-02-20T12:55:05.211 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:05.209+0000 7f2c50afc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2c4c0f5480 0x7f2c4c1056f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:55:05.211 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:05.209+0000 7f2c50afc700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c4c105be0 con 0x7f2c4c0f5480 2024-02-20T12:55:05.211 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:05.209+0000 7f2c4a59c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2c4c0f5480 0x7f2c4c1056f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:55:05.211 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:05.209+0000 7f2c4a59c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2c4c0f5480 0x7f2c4c1056f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.71:45564/0 (socket says 172.21.15.71:45564) 2024-02-20T12:55:05.211 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:05.209+0000 7f2c4a59c700 1 -- 172.21.15.71:0/3874890096 learned_addr learned my addr 172.21.15.71:0/3874890096 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-02-20T12:55:05.211 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:05.209+0000 7f2c4a59c700 1 -- 172.21.15.71:0/3874890096 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c3c005040 con 0x7f2c4c0f5480 2024-02-20T12:55:05.211 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:05.209+0000 7f2c4a59c700 1 --2- 172.21.15.71:0/3874890096 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2c4c0f5480 0x7f2c4c1056f0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 rx=0x7f2c3c000c00 tx=0x7f2c3c00b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:55:05.211 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:05.209+0000 7f2c3affd700 1 -- 172.21.15.71:0/3874890096 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2c3c0095f0 con 0x7f2c4c0f5480 2024-02-20T12:55:05.211 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:05.210+0000 7f2c50afc700 1 -- 172.21.15.71:0/3874890096 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c4c105de0 con 0x7f2c4c0f5480 2024-02-20T12:55:05.211 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:05.210+0000 7f2c3affd700 1 -- 172.21.15.71:0/3874890096 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(25 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f2c3c007e50 con 0x7f2c4c0f5480 2024-02-20T12:55:05.211 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:05.210+0000 7f2c3affd700 1 -- 172.21.15.71:0/3874890096 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2c3c014e70 con 0x7f2c4c0f5480 2024-02-20T12:55:05.212 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:05.210+0000 7f2c50afc700 1 -- 172.21.15.71:0/3874890096 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c4c106270 con 0x7f2c4c0f5480 2024-02-20T12:55:05.212 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:05.210+0000 7f2c3affd700 1 -- 172.21.15.71:0/3874890096 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 37151+0+0 (secure 0 0 0) 0x7f2c3c025070 con 0x7f2c4c0f5480 2024-02-20T12:55:05.213 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:05.210+0000 7f2c3affd700 1 --2- 172.21.15.71:0/3874890096 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f2c34031090 0x7f2c34033550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:55:05.213 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:05.210+0000 7f2c3affd700 1 -- 172.21.15.71:0/3874890096 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1065+0+0 (secure 0 0 0) 0x7f2c3c041d10 con 0x7f2c4c0f5480 2024-02-20T12:55:05.213 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:05.210+0000 7f2c50afc700 1 -- 172.21.15.71:0/3874890096 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c4c045bb0 con 0x7f2c4c0f5480 2024-02-20T12:55:05.213 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:05.211+0000 7f2c49d9b700 1 --2- 172.21.15.71:0/3874890096 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f2c34031090 0x7f2c34033550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:55:05.213 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:05.211+0000 7f2c49d9b700 1 --2- 172.21.15.71:0/3874890096 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f2c34031090 0x7f2c34033550 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 rx=0x7f2c400096a0 tx=0x7f2c40006b40).ready entity=mgr.14156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:55:05.215 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:05.215+0000 7f2c3affd700 1 -- 172.21.15.71:0/3874890096 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+153237 (secure 0 0 0) 0x7f2c3c0079c0 con 0x7f2c4c0f5480 2024-02-20T12:55:05.422 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:05.421+0000 7f2c50afc700 1 -- 172.21.15.71:0/3874890096 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f2c4c0fb060 con 0x7f2c4c0f5480 2024-02-20T12:55:05.422 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:05.422+0000 7f2c3affd700 1 -- 172.21.15.71:0/3874890096 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7f2c3c02d090 con 0x7f2c4c0f5480 2024-02-20T12:55:05.423 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-20T12:55:05.423 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":1,"fsid":"680ee126-cfee-11ee-95bb-87774f69a715","modified":"2024-02-20T12:49:44.699059Z","created":"2024-02-20T12:49:44.699059Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi064","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:3300","nonce":0},{"type":"v1","addr":"172.21.15.64:6789","nonce":0}]},"addr":"172.21.15.64:6789/0","public_addr":"172.21.15.64:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-20T12:55:05.424 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:05.424+0000 7f2c50afc700 1 -- 172.21.15.71:0/3874890096 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f2c34031090 msgr2=0x7f2c34033550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:55:05.424 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:05.424+0000 7f2c50afc700 1 --2- 172.21.15.71:0/3874890096 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f2c34031090 0x7f2c34033550 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 rx=0x7f2c400096a0 tx=0x7f2c40006b40).stop 2024-02-20T12:55:05.425 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:05.424+0000 7f2c50afc700 1 -- 172.21.15.71:0/3874890096 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2c4c0f5480 msgr2=0x7f2c4c1056f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:55:05.425 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:05.424+0000 7f2c50afc700 1 --2- 172.21.15.71:0/3874890096 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2c4c0f5480 0x7f2c4c1056f0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 rx=0x7f2c3c000c00 tx=0x7f2c3c00b040).stop 2024-02-20T12:55:05.425 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:05.424+0000 7f2c50afc700 1 -- 172.21.15.71:0/3874890096 shutdown_connections 2024-02-20T12:55:05.425 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:05.424+0000 7f2c50afc700 1 --2- 172.21.15.71:0/3874890096 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f2c34031090 0x7f2c34033550 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:55:05.425 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:05.424+0000 7f2c50afc700 1 --2- 172.21.15.71:0/3874890096 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2c4c0f5480 0x7f2c4c1056f0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:55:05.425 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:05.424+0000 7f2c50afc700 1 -- 172.21.15.71:0/3874890096 >> 172.21.15.71:0/3874890096 conn(0x7f2c4c0f0be0 msgr2=0x7f2c4c0f16d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:55:05.425 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:05.424+0000 7f2c50afc700 1 -- 172.21.15.71:0/3874890096 shutdown_connections 2024-02-20T12:55:05.425 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:05.424+0000 7f2c50afc700 1 -- 172.21.15.71:0/3874890096 wait complete. 2024-02-20T12:55:05.425 INFO:teuthology.orchestra.run.smithi071.stderr:dumped monmap epoch 1 2024-02-20T12:55:05.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:55:05 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:55:03.742453+0000 mgr.smithi064.vriyor (mgr.14156) 127 : cluster [DBG] pgmap v119: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:55:06.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:55:06 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:55:05.422915+0000 mon.smithi064 (mon.0) 172 : audit [DBG] from='client.? 172.21.15.71:0/3874890096' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-20T12:55:07.098 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-20T12:55:07.098 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/daemon-base:latest-pacific shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 680ee126-cfee-11ee-95bb-87774f69a715 -- ceph mon dump -f json 2024-02-20T12:55:07.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:55:07 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:55:05.742769+0000 mgr.smithi064.vriyor (mgr.14156) 128 : cluster [DBG] pgmap v120: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:55:08.982 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:08.948+0000 7f32ec4f1700 1 -- 172.21.15.71:0/3282692415 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f32e40f5480 msgr2=0x7f32e40f58a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:55:08.982 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:08.948+0000 7f32ec4f1700 1 --2- 172.21.15.71:0/3282692415 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f32e40f5480 0x7f32e40f58a0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 rx=0x7f32e0009fb0 tx=0x7f32e0009b30).stop 2024-02-20T12:55:08.982 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:08.948+0000 7f32ec4f1700 1 -- 172.21.15.71:0/3282692415 shutdown_connections 2024-02-20T12:55:08.982 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:08.948+0000 7f32ec4f1700 1 --2- 172.21.15.71:0/3282692415 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f32e40f5480 0x7f32e40f58a0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:55:08.982 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:08.948+0000 7f32ec4f1700 1 -- 172.21.15.71:0/3282692415 >> 172.21.15.71:0/3282692415 conn(0x7f32e40f0be0 msgr2=0x7f32e40f3000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:55:08.982 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:08.948+0000 7f32ec4f1700 1 -- 172.21.15.71:0/3282692415 shutdown_connections 2024-02-20T12:55:08.982 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:08.948+0000 7f32ec4f1700 1 -- 172.21.15.71:0/3282692415 wait complete. 2024-02-20T12:55:08.982 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:08.948+0000 7f32ec4f1700 1 Processor -- start 2024-02-20T12:55:08.982 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:08.949+0000 7f32ec4f1700 1 -- start start 2024-02-20T12:55:08.983 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:08.949+0000 7f32ec4f1700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f32e40f5480 0x7f32e41067e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:55:08.983 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:08.949+0000 7f32ec4f1700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32e4106cd0 con 0x7f32e40f5480 2024-02-20T12:55:08.983 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:08.949+0000 7f32ea28d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f32e40f5480 0x7f32e41067e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:55:08.983 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:08.949+0000 7f32ea28d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f32e40f5480 0x7f32e41067e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.71:35016/0 (socket says 172.21.15.71:35016) 2024-02-20T12:55:08.983 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:08.949+0000 7f32ea28d700 1 -- 172.21.15.71:0/2624060882 learned_addr learned my addr 172.21.15.71:0/2624060882 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-02-20T12:55:08.983 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:08.949+0000 7f32ea28d700 1 -- 172.21.15.71:0/2624060882 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32e0005040 con 0x7f32e40f5480 2024-02-20T12:55:08.983 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:08.949+0000 7f32ea28d700 1 --2- 172.21.15.71:0/2624060882 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f32e40f5480 0x7f32e41067e0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 rx=0x7f32e00089f0 tx=0x7f32e000b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:55:08.983 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:08.949+0000 7f32daffd700 1 -- 172.21.15.71:0/2624060882 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f32e0009320 con 0x7f32e40f5480 2024-02-20T12:55:08.983 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:08.949+0000 7f32ec4f1700 1 -- 172.21.15.71:0/2624060882 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32e4106ed0 con 0x7f32e40f5480 2024-02-20T12:55:08.983 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:08.949+0000 7f32daffd700 1 -- 172.21.15.71:0/2624060882 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(25 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f32e0013ec0 con 0x7f32e40f5480 2024-02-20T12:55:08.983 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:08.949+0000 7f32daffd700 1 -- 172.21.15.71:0/2624060882 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f32e0014d30 con 0x7f32e40f5480 2024-02-20T12:55:08.983 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:08.950+0000 7f32ec4f1700 1 -- 172.21.15.71:0/2624060882 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32e4107360 con 0x7f32e40f5480 2024-02-20T12:55:08.983 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:08.950+0000 7f32daffd700 1 -- 172.21.15.71:0/2624060882 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 37151+0+0 (secure 0 0 0) 0x7f32e0014420 con 0x7f32e40f5480 2024-02-20T12:55:08.983 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:08.950+0000 7f32daffd700 1 --2- 172.21.15.71:0/2624060882 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f32d0031090 0x7f32d0033550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:55:08.983 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:08.950+0000 7f32ec4f1700 1 -- 172.21.15.71:0/2624060882 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32e4045bb0 con 0x7f32e40f5480 2024-02-20T12:55:08.984 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:08.950+0000 7f32daffd700 1 -- 172.21.15.71:0/2624060882 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1065+0+0 (secure 0 0 0) 0x7f32e0042480 con 0x7f32e40f5480 2024-02-20T12:55:08.984 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:08.951+0000 7f32e9a8c700 1 --2- 172.21.15.71:0/2624060882 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f32d0031090 0x7f32d0033550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:55:08.984 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:08.951+0000 7f32e9a8c700 1 --2- 172.21.15.71:0/2624060882 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f32d0031090 0x7f32d0033550 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 rx=0x7f32d40096a0 tx=0x7f32d4006b40).ready entity=mgr.14156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:55:08.984 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:08.955+0000 7f32daffd700 1 -- 172.21.15.71:0/2624060882 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+153237 (secure 0 0 0) 0x7f32e001d980 con 0x7f32e40f5480 2024-02-20T12:55:09.161 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:09.161+0000 7f32ec4f1700 1 -- 172.21.15.71:0/2624060882 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f32e4108160 con 0x7f32e40f5480 2024-02-20T12:55:09.162 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:09.161+0000 7f32daffd700 1 -- 172.21.15.71:0/2624060882 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7f32e002c090 con 0x7f32e40f5480 2024-02-20T12:55:09.162 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-20T12:55:09.162 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":1,"fsid":"680ee126-cfee-11ee-95bb-87774f69a715","modified":"2024-02-20T12:49:44.699059Z","created":"2024-02-20T12:49:44.699059Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi064","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:3300","nonce":0},{"type":"v1","addr":"172.21.15.64:6789","nonce":0}]},"addr":"172.21.15.64:6789/0","public_addr":"172.21.15.64:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-20T12:55:09.163 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:09.163+0000 7f32ec4f1700 1 -- 172.21.15.71:0/2624060882 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f32d0031090 msgr2=0x7f32d0033550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:55:09.163 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:09.163+0000 7f32ec4f1700 1 --2- 172.21.15.71:0/2624060882 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f32d0031090 0x7f32d0033550 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 rx=0x7f32d40096a0 tx=0x7f32d4006b40).stop 2024-02-20T12:55:09.163 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:09.163+0000 7f32ec4f1700 1 -- 172.21.15.71:0/2624060882 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f32e40f5480 msgr2=0x7f32e41067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:55:09.163 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:09.163+0000 7f32ec4f1700 1 --2- 172.21.15.71:0/2624060882 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f32e40f5480 0x7f32e41067e0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 rx=0x7f32e00089f0 tx=0x7f32e000b040).stop 2024-02-20T12:55:09.164 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:09.163+0000 7f32ec4f1700 1 -- 172.21.15.71:0/2624060882 shutdown_connections 2024-02-20T12:55:09.164 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:09.163+0000 7f32ec4f1700 1 --2- 172.21.15.71:0/2624060882 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f32d0031090 0x7f32d0033550 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:55:09.164 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:09.163+0000 7f32ec4f1700 1 --2- 172.21.15.71:0/2624060882 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f32e40f5480 0x7f32e41067e0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:55:09.165 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:09.163+0000 7f32ec4f1700 1 -- 172.21.15.71:0/2624060882 >> 172.21.15.71:0/2624060882 conn(0x7f32e40f0be0 msgr2=0x7f32e405ff40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:55:09.165 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:09.163+0000 7f32ec4f1700 1 -- 172.21.15.71:0/2624060882 shutdown_connections 2024-02-20T12:55:09.165 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:09.163+0000 7f32ec4f1700 1 -- 172.21.15.71:0/2624060882 wait complete. 2024-02-20T12:55:09.166 INFO:teuthology.orchestra.run.smithi071.stderr:dumped monmap epoch 1 2024-02-20T12:55:09.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:55:09 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:55:07.743039+0000 mgr.smithi064.vriyor (mgr.14156) 129 : cluster [DBG] pgmap v121: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:55:10.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:55:10 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:55:09.161984+0000 mon.smithi064 (mon.0) 173 : audit [DBG] from='client.? 172.21.15.71:0/2624060882' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-20T12:55:10.867 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-20T12:55:10.867 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/daemon-base:latest-pacific shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 680ee126-cfee-11ee-95bb-87774f69a715 -- ceph mon dump -f json 2024-02-20T12:55:11.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:55:11 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:55:09.743240+0000 mgr.smithi064.vriyor (mgr.14156) 130 : cluster [DBG] pgmap v122: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:55:12.750 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:12.748+0000 7f88515e0700 1 -- 172.21.15.71:0/1455484436 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f884c0f52d0 msgr2=0x7f884c0f56f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:55:12.750 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:12.748+0000 7f88515e0700 1 --2- 172.21.15.71:0/1455484436 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f884c0f52d0 0x7f884c0f56f0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 rx=0x7f883c009fb0 tx=0x7f883c009b30).stop 2024-02-20T12:55:12.750 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:12.748+0000 7f88515e0700 1 -- 172.21.15.71:0/1455484436 shutdown_connections 2024-02-20T12:55:12.750 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:12.748+0000 7f88515e0700 1 --2- 172.21.15.71:0/1455484436 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f884c0f52d0 0x7f884c0f56f0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:55:12.750 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:12.748+0000 7f88515e0700 1 -- 172.21.15.71:0/1455484436 >> 172.21.15.71:0/1455484436 conn(0x7f884c0f0a30 msgr2=0x7f884c0f2e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:55:12.750 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:12.748+0000 7f88515e0700 1 -- 172.21.15.71:0/1455484436 shutdown_connections 2024-02-20T12:55:12.750 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:12.748+0000 7f88515e0700 1 -- 172.21.15.71:0/1455484436 wait complete. 2024-02-20T12:55:12.750 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:12.748+0000 7f88515e0700 1 Processor -- start 2024-02-20T12:55:12.750 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:12.748+0000 7f88515e0700 1 -- start start 2024-02-20T12:55:12.750 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:12.749+0000 7f88515e0700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f884c0f52d0 0x7f884c105510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:55:12.750 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:12.749+0000 7f88515e0700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f884c105a00 con 0x7f884c0f52d0 2024-02-20T12:55:12.751 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:12.749+0000 7f884bfff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f884c0f52d0 0x7f884c105510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:55:12.751 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:12.749+0000 7f884bfff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f884c0f52d0 0x7f884c105510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.71:35022/0 (socket says 172.21.15.71:35022) 2024-02-20T12:55:12.751 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:12.749+0000 7f884bfff700 1 -- 172.21.15.71:0/4180764667 learned_addr learned my addr 172.21.15.71:0/4180764667 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-02-20T12:55:12.751 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:12.749+0000 7f884bfff700 1 -- 172.21.15.71:0/4180764667 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f883c005040 con 0x7f884c0f52d0 2024-02-20T12:55:12.751 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:12.749+0000 7f884bfff700 1 --2- 172.21.15.71:0/4180764667 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f884c0f52d0 0x7f884c105510 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 rx=0x7f883c0089f0 tx=0x7f883c00b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:55:12.751 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:12.749+0000 7f8848ff9700 1 -- 172.21.15.71:0/4180764667 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f883c009320 con 0x7f884c0f52d0 2024-02-20T12:55:12.751 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:12.749+0000 7f88515e0700 1 -- 172.21.15.71:0/4180764667 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f884c105c00 con 0x7f884c0f52d0 2024-02-20T12:55:12.751 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:12.749+0000 7f8848ff9700 1 -- 172.21.15.71:0/4180764667 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(25 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f883c013ec0 con 0x7f884c0f52d0 2024-02-20T12:55:12.751 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:12.749+0000 7f8848ff9700 1 -- 172.21.15.71:0/4180764667 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f883c014d30 con 0x7f884c0f52d0 2024-02-20T12:55:12.751 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:12.749+0000 7f88515e0700 1 -- 172.21.15.71:0/4180764667 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f884c106090 con 0x7f884c0f52d0 2024-02-20T12:55:12.752 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:12.750+0000 7f8848ff9700 1 -- 172.21.15.71:0/4180764667 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 37151+0+0 (secure 0 0 0) 0x7f883c0095f0 con 0x7f884c0f52d0 2024-02-20T12:55:12.752 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:12.750+0000 7f88515e0700 1 -- 172.21.15.71:0/4180764667 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f884c045bb0 con 0x7f884c0f52d0 2024-02-20T12:55:12.752 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:12.750+0000 7f8848ff9700 1 --2- 172.21.15.71:0/4180764667 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f8834031090 0x7f8834033550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:55:12.752 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:12.751+0000 7f8848ff9700 1 -- 172.21.15.71:0/4180764667 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1065+0+0 (secure 0 0 0) 0x7f883c042440 con 0x7f884c0f52d0 2024-02-20T12:55:12.753 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:12.751+0000 7f884b7fe700 1 --2- 172.21.15.71:0/4180764667 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f8834031090 0x7f8834033550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:55:12.753 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:12.751+0000 7f884b7fe700 1 --2- 172.21.15.71:0/4180764667 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f8834031090 0x7f8834033550 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 rx=0x7f88400096a0 tx=0x7f8840006b40).ready entity=mgr.14156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:55:12.755 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:12.755+0000 7f8848ff9700 1 -- 172.21.15.71:0/4180764667 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+153237 (secure 0 0 0) 0x7f883c01d7f0 con 0x7f884c0f52d0 2024-02-20T12:55:12.960 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:12.959+0000 7f88515e0700 1 -- 172.21.15.71:0/4180764667 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f884c106ee0 con 0x7f884c0f52d0 2024-02-20T12:55:12.961 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:12.960+0000 7f8848ff9700 1 -- 172.21.15.71:0/4180764667 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7f883c01d7f0 con 0x7f884c0f52d0 2024-02-20T12:55:12.961 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-20T12:55:12.961 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":1,"fsid":"680ee126-cfee-11ee-95bb-87774f69a715","modified":"2024-02-20T12:49:44.699059Z","created":"2024-02-20T12:49:44.699059Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi064","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:3300","nonce":0},{"type":"v1","addr":"172.21.15.64:6789","nonce":0}]},"addr":"172.21.15.64:6789/0","public_addr":"172.21.15.64:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-20T12:55:12.963 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:12.962+0000 7f88515e0700 1 -- 172.21.15.71:0/4180764667 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f8834031090 msgr2=0x7f8834033550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:55:12.963 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:12.962+0000 7f88515e0700 1 --2- 172.21.15.71:0/4180764667 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f8834031090 0x7f8834033550 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 rx=0x7f88400096a0 tx=0x7f8840006b40).stop 2024-02-20T12:55:12.963 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:12.962+0000 7f88515e0700 1 -- 172.21.15.71:0/4180764667 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f884c0f52d0 msgr2=0x7f884c105510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:55:12.963 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:12.962+0000 7f88515e0700 1 --2- 172.21.15.71:0/4180764667 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f884c0f52d0 0x7f884c105510 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 rx=0x7f883c0089f0 tx=0x7f883c00b040).stop 2024-02-20T12:55:12.963 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:12.962+0000 7f88515e0700 1 -- 172.21.15.71:0/4180764667 shutdown_connections 2024-02-20T12:55:12.963 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:12.962+0000 7f88515e0700 1 --2- 172.21.15.71:0/4180764667 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f8834031090 0x7f8834033550 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:55:12.963 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:12.962+0000 7f88515e0700 1 --2- 172.21.15.71:0/4180764667 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f884c0f52d0 0x7f884c105510 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:55:12.963 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:12.962+0000 7f88515e0700 1 -- 172.21.15.71:0/4180764667 >> 172.21.15.71:0/4180764667 conn(0x7f884c0f0a30 msgr2=0x7f884c0f1550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:55:12.963 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:12.962+0000 7f88515e0700 1 -- 172.21.15.71:0/4180764667 shutdown_connections 2024-02-20T12:55:12.963 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:12.962+0000 7f88515e0700 1 -- 172.21.15.71:0/4180764667 wait complete. 2024-02-20T12:55:12.963 INFO:teuthology.orchestra.run.smithi071.stderr:dumped monmap epoch 1 2024-02-20T12:55:13.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:55:13 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:55:11.743573+0000 mgr.smithi064.vriyor (mgr.14156) 131 : cluster [DBG] pgmap v123: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:55:13.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:55:13 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:55:12.960878+0000 mon.smithi064 (mon.0) 174 : audit [DBG] from='client.? 172.21.15.71:0/4180764667' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-20T12:55:14.559 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-20T12:55:14.559 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/daemon-base:latest-pacific shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 680ee126-cfee-11ee-95bb-87774f69a715 -- ceph mon dump -f json 2024-02-20T12:55:15.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:55:15 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:55:13.743975+0000 mgr.smithi064.vriyor (mgr.14156) 132 : cluster [DBG] pgmap v124: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:55:16.373 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:16.371+0000 7f26831f5700 1 -- 172.21.15.71:0/393022451 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f267c0f6590 msgr2=0x7f267c0f69b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:55:16.373 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:16.371+0000 7f26831f5700 1 --2- 172.21.15.71:0/393022451 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f267c0f6590 0x7f267c0f69b0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 rx=0x7f2670004760 tx=0x7f2670009ad0).stop 2024-02-20T12:55:16.374 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:16.371+0000 7f26831f5700 1 -- 172.21.15.71:0/393022451 shutdown_connections 2024-02-20T12:55:16.374 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:16.371+0000 7f26831f5700 1 --2- 172.21.15.71:0/393022451 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f267c0f6590 0x7f267c0f69b0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:55:16.374 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:16.371+0000 7f26831f5700 1 -- 172.21.15.71:0/393022451 >> 172.21.15.71:0/393022451 conn(0x7f267c0f1cd0 msgr2=0x7f267c0f4110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:55:16.374 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:16.371+0000 7f26831f5700 1 -- 172.21.15.71:0/393022451 shutdown_connections 2024-02-20T12:55:16.374 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:16.371+0000 7f26831f5700 1 -- 172.21.15.71:0/393022451 wait complete. 2024-02-20T12:55:16.374 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:16.372+0000 7f26831f5700 1 Processor -- start 2024-02-20T12:55:16.374 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:16.372+0000 7f26831f5700 1 -- start start 2024-02-20T12:55:16.374 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:16.372+0000 7f26831f5700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f267c0f6590 0x7f267c1067c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:55:16.374 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:16.372+0000 7f26831f5700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f267c106cb0 con 0x7f267c0f6590 2024-02-20T12:55:16.374 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:16.372+0000 7f2680f91700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f267c0f6590 0x7f267c1067c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:55:16.374 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:16.372+0000 7f2680f91700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f267c0f6590 0x7f267c1067c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.71:35050/0 (socket says 172.21.15.71:35050) 2024-02-20T12:55:16.374 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:16.372+0000 7f2680f91700 1 -- 172.21.15.71:0/2590994900 learned_addr learned my addr 172.21.15.71:0/2590994900 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-02-20T12:55:16.375 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:16.373+0000 7f2680f91700 1 -- 172.21.15.71:0/2590994900 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2670009d70 con 0x7f267c0f6590 2024-02-20T12:55:16.375 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:16.373+0000 7f2680f91700 1 --2- 172.21.15.71:0/2590994900 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f267c0f6590 0x7f267c1067c0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 rx=0x7f2670004170 tx=0x7f2670009ad0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:55:16.375 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:16.373+0000 7f26797fa700 1 -- 172.21.15.71:0/2590994900 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f267000e040 con 0x7f267c0f6590 2024-02-20T12:55:16.375 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:16.373+0000 7f26831f5700 1 -- 172.21.15.71:0/2590994900 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f267c106eb0 con 0x7f267c0f6590 2024-02-20T12:55:16.375 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:16.373+0000 7f26797fa700 1 -- 172.21.15.71:0/2590994900 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(25 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f2670008ef0 con 0x7f267c0f6590 2024-02-20T12:55:16.375 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:16.373+0000 7f26797fa700 1 -- 172.21.15.71:0/2590994900 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2670014c90 con 0x7f267c0f6590 2024-02-20T12:55:16.375 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:16.373+0000 7f26831f5700 1 -- 172.21.15.71:0/2590994900 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f267c107340 con 0x7f267c0f6590 2024-02-20T12:55:16.376 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:16.374+0000 7f26797fa700 1 -- 172.21.15.71:0/2590994900 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 37151+0+0 (secure 0 0 0) 0x7f2670014420 con 0x7f267c0f6590 2024-02-20T12:55:16.376 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:16.374+0000 7f26797fa700 1 --2- 172.21.15.71:0/2590994900 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f266c031090 0x7f266c033550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:55:16.376 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:16.374+0000 7f26831f5700 1 -- 172.21.15.71:0/2590994900 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f267c045bb0 con 0x7f267c0f6590 2024-02-20T12:55:16.377 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:16.374+0000 7f26797fa700 1 -- 172.21.15.71:0/2590994900 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1065+0+0 (secure 0 0 0) 0x7f2670042220 con 0x7f267c0f6590 2024-02-20T12:55:16.377 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:16.374+0000 7f267bfff700 1 --2- 172.21.15.71:0/2590994900 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f266c031090 0x7f266c033550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:55:16.377 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:16.374+0000 7f267bfff700 1 --2- 172.21.15.71:0/2590994900 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f266c031090 0x7f266c033550 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 rx=0x7f26680096a0 tx=0x7f2668006b40).ready entity=mgr.14156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:55:16.379 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:16.379+0000 7f26797fa700 1 -- 172.21.15.71:0/2590994900 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+153237 (secure 0 0 0) 0x7f2670012d00 con 0x7f267c0f6590 2024-02-20T12:55:16.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:16.586+0000 7f26831f5700 1 -- 172.21.15.71:0/2590994900 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f267c0fc080 con 0x7f267c0f6590 2024-02-20T12:55:16.588 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:16.587+0000 7f26797fa700 1 -- 172.21.15.71:0/2590994900 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7f2670012b20 con 0x7f267c0f6590 2024-02-20T12:55:16.588 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-20T12:55:16.588 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":1,"fsid":"680ee126-cfee-11ee-95bb-87774f69a715","modified":"2024-02-20T12:49:44.699059Z","created":"2024-02-20T12:49:44.699059Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi064","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:3300","nonce":0},{"type":"v1","addr":"172.21.15.64:6789","nonce":0}]},"addr":"172.21.15.64:6789/0","public_addr":"172.21.15.64:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-20T12:55:16.590 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:16.588+0000 7f26831f5700 1 -- 172.21.15.71:0/2590994900 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f266c031090 msgr2=0x7f266c033550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:55:16.590 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:16.588+0000 7f26831f5700 1 --2- 172.21.15.71:0/2590994900 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f266c031090 0x7f266c033550 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 rx=0x7f26680096a0 tx=0x7f2668006b40).stop 2024-02-20T12:55:16.590 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:16.588+0000 7f26831f5700 1 -- 172.21.15.71:0/2590994900 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f267c0f6590 msgr2=0x7f267c1067c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:55:16.590 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:16.588+0000 7f26831f5700 1 --2- 172.21.15.71:0/2590994900 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f267c0f6590 0x7f267c1067c0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 rx=0x7f2670004170 tx=0x7f2670009ad0).stop 2024-02-20T12:55:16.590 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:16.588+0000 7f26831f5700 1 -- 172.21.15.71:0/2590994900 shutdown_connections 2024-02-20T12:55:16.590 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:16.588+0000 7f26831f5700 1 --2- 172.21.15.71:0/2590994900 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f266c031090 0x7f266c033550 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:55:16.591 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:16.588+0000 7f26831f5700 1 --2- 172.21.15.71:0/2590994900 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f267c0f6590 0x7f267c1067c0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:55:16.591 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:16.588+0000 7f26831f5700 1 -- 172.21.15.71:0/2590994900 >> 172.21.15.71:0/2590994900 conn(0x7f267c0f1cd0 msgr2=0x7f267c0f27c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:55:16.591 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:16.589+0000 7f26831f5700 1 -- 172.21.15.71:0/2590994900 shutdown_connections 2024-02-20T12:55:16.591 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:16.589+0000 7f26831f5700 1 -- 172.21.15.71:0/2590994900 wait complete. 2024-02-20T12:55:16.591 INFO:teuthology.orchestra.run.smithi071.stderr:dumped monmap epoch 1 2024-02-20T12:55:17.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:55:17 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:55:15.744360+0000 mgr.smithi064.vriyor (mgr.14156) 133 : cluster [DBG] pgmap v125: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:55:17.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:55:17 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:55:16.587737+0000 mon.smithi064 (mon.0) 175 : audit [DBG] from='client.? 172.21.15.71:0/2590994900' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-20T12:55:18.270 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-20T12:55:18.271 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/daemon-base:latest-pacific shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 680ee126-cfee-11ee-95bb-87774f69a715 -- ceph mon dump -f json 2024-02-20T12:55:19.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:55:19 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:55:17.744680+0000 mgr.smithi064.vriyor (mgr.14156) 134 : cluster [DBG] pgmap v126: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:55:20.207 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:20.194+0000 7f957778a700 1 -- 172.21.15.71:0/755702820 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9570060360 msgr2=0x7f95700f90f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:55:20.208 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:20.194+0000 7f957778a700 1 --2- 172.21.15.71:0/755702820 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9570060360 0x7f95700f90f0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 rx=0x7f956c009fb0 tx=0x7f956c009b30).stop 2024-02-20T12:55:20.208 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:20.194+0000 7f957778a700 1 -- 172.21.15.71:0/755702820 shutdown_connections 2024-02-20T12:55:20.208 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:20.194+0000 7f957778a700 1 --2- 172.21.15.71:0/755702820 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9570060360 0x7f95700f90f0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:55:20.208 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:20.194+0000 7f957778a700 1 -- 172.21.15.71:0/755702820 >> 172.21.15.71:0/755702820 conn(0x7f95700f0b70 msgr2=0x7f95700f2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:55:20.208 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:20.194+0000 7f957778a700 1 -- 172.21.15.71:0/755702820 shutdown_connections 2024-02-20T12:55:20.208 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:20.194+0000 7f957778a700 1 -- 172.21.15.71:0/755702820 wait complete. 2024-02-20T12:55:20.208 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:20.195+0000 7f957778a700 1 Processor -- start 2024-02-20T12:55:20.208 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:20.195+0000 7f957778a700 1 -- start start 2024-02-20T12:55:20.208 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:20.195+0000 7f957778a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9570060360 0x7f95700f6b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:55:20.208 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:20.195+0000 7f957778a700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95700f4ff0 con 0x7f9570060360 2024-02-20T12:55:20.208 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:20.195+0000 7f9575526700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9570060360 0x7f95700f6b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:55:20.208 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:20.195+0000 7f9575526700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9570060360 0x7f95700f6b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.71:35634/0 (socket says 172.21.15.71:35634) 2024-02-20T12:55:20.209 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:20.195+0000 7f9575526700 1 -- 172.21.15.71:0/1115365127 learned_addr learned my addr 172.21.15.71:0/1115365127 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-02-20T12:55:20.209 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:20.195+0000 7f9575526700 1 -- 172.21.15.71:0/1115365127 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f956c005040 con 0x7f9570060360 2024-02-20T12:55:20.209 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:20.195+0000 7f9575526700 1 --2- 172.21.15.71:0/1115365127 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9570060360 0x7f95700f6b70 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 rx=0x7f956c009f80 tx=0x7f956c00b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:55:20.209 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:20.196+0000 7f9565ffb700 1 -- 172.21.15.71:0/1115365127 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f956c009320 con 0x7f9570060360 2024-02-20T12:55:20.209 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:20.196+0000 7f9565ffb700 1 -- 172.21.15.71:0/1115365127 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(25 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f956c013680 con 0x7f9570060360 2024-02-20T12:55:20.209 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:20.196+0000 7f957778a700 1 -- 172.21.15.71:0/1115365127 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95700f5190 con 0x7f9570060360 2024-02-20T12:55:20.209 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:20.196+0000 7f9565ffb700 1 -- 172.21.15.71:0/1115365127 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f956c01eda0 con 0x7f9570060360 2024-02-20T12:55:20.209 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:20.196+0000 7f957778a700 1 -- 172.21.15.71:0/1115365127 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95700f5620 con 0x7f9570060360 2024-02-20T12:55:20.209 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:20.197+0000 7f9565ffb700 1 -- 172.21.15.71:0/1115365127 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 37151+0+0 (secure 0 0 0) 0x7f956c01e420 con 0x7f9570060360 2024-02-20T12:55:20.209 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:20.197+0000 7f957778a700 1 -- 172.21.15.71:0/1115365127 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9570045bb0 con 0x7f9570060360 2024-02-20T12:55:20.210 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:20.197+0000 7f9565ffb700 1 --2- 172.21.15.71:0/1115365127 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f955c0310e0 0x7f955c0335a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:55:20.210 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:20.197+0000 7f9565ffb700 1 -- 172.21.15.71:0/1115365127 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1065+0+0 (secure 0 0 0) 0x7f956c042350 con 0x7f9570060360 2024-02-20T12:55:20.210 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:20.197+0000 7f9574d25700 1 --2- 172.21.15.71:0/1115365127 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f955c0310e0 0x7f955c0335a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:55:20.210 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:20.198+0000 7f9574d25700 1 --2- 172.21.15.71:0/1115365127 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f955c0310e0 0x7f955c0335a0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 rx=0x7f95600096a0 tx=0x7f9560006b40).ready entity=mgr.14156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:55:20.210 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:20.202+0000 7f9565ffb700 1 -- 172.21.15.71:0/1115365127 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+153237 (secure 0 0 0) 0x7f956c01e6b0 con 0x7f9570060360 2024-02-20T12:55:20.408 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:20.407+0000 7f957778a700 1 -- 172.21.15.71:0/1115365127 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f957005eb30 con 0x7f9570060360 2024-02-20T12:55:20.408 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:20.408+0000 7f9565ffb700 1 -- 172.21.15.71:0/1115365127 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7f956c02c2d0 con 0x7f9570060360 2024-02-20T12:55:20.409 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-20T12:55:20.409 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":1,"fsid":"680ee126-cfee-11ee-95bb-87774f69a715","modified":"2024-02-20T12:49:44.699059Z","created":"2024-02-20T12:49:44.699059Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi064","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:3300","nonce":0},{"type":"v1","addr":"172.21.15.64:6789","nonce":0}]},"addr":"172.21.15.64:6789/0","public_addr":"172.21.15.64:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-20T12:55:20.411 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:20.409+0000 7f957778a700 1 -- 172.21.15.71:0/1115365127 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f955c0310e0 msgr2=0x7f955c0335a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:55:20.411 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:20.409+0000 7f957778a700 1 --2- 172.21.15.71:0/1115365127 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f955c0310e0 0x7f955c0335a0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 rx=0x7f95600096a0 tx=0x7f9560006b40).stop 2024-02-20T12:55:20.411 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:20.409+0000 7f957778a700 1 -- 172.21.15.71:0/1115365127 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9570060360 msgr2=0x7f95700f6b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:55:20.412 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:20.409+0000 7f957778a700 1 --2- 172.21.15.71:0/1115365127 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9570060360 0x7f95700f6b70 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 rx=0x7f956c009f80 tx=0x7f956c00b040).stop 2024-02-20T12:55:20.412 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:20.409+0000 7f957778a700 1 -- 172.21.15.71:0/1115365127 shutdown_connections 2024-02-20T12:55:20.412 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:20.409+0000 7f957778a700 1 --2- 172.21.15.71:0/1115365127 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f955c0310e0 0x7f955c0335a0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:55:20.412 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:20.409+0000 7f957778a700 1 --2- 172.21.15.71:0/1115365127 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9570060360 0x7f95700f6b70 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:55:20.412 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:20.409+0000 7f957778a700 1 -- 172.21.15.71:0/1115365127 >> 172.21.15.71:0/1115365127 conn(0x7f95700f0b70 msgr2=0x7f95700f1650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:55:20.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:20.409+0000 7f957778a700 1 -- 172.21.15.71:0/1115365127 shutdown_connections 2024-02-20T12:55:20.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:20.409+0000 7f957778a700 1 -- 172.21.15.71:0/1115365127 wait complete. 2024-02-20T12:55:20.413 INFO:teuthology.orchestra.run.smithi071.stderr:dumped monmap epoch 1 2024-02-20T12:55:21.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:55:21 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:55:19.744960+0000 mgr.smithi064.vriyor (mgr.14156) 135 : cluster [DBG] pgmap v127: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:55:21.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:55:21 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:55:20.408484+0000 mon.smithi064 (mon.0) 176 : audit [DBG] from='client.? 172.21.15.71:0/1115365127' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-20T12:55:22.155 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-20T12:55:22.156 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/daemon-base:latest-pacific shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 680ee126-cfee-11ee-95bb-87774f69a715 -- ceph mon dump -f json 2024-02-20T12:55:23.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:55:23 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:55:21.745299+0000 mgr.smithi064.vriyor (mgr.14156) 136 : cluster [DBG] pgmap v128: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:55:24.017 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:24.015+0000 7f71b1611700 1 -- 172.21.15.71:0/1663845173 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f71ac0f54b0 msgr2=0x7f71ac0f58d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:55:24.017 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:24.015+0000 7f71b1611700 1 --2- 172.21.15.71:0/1663845173 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f71ac0f54b0 0x7f71ac0f58d0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 rx=0x7f71a0009fb0 tx=0x7f71a0009b30).stop 2024-02-20T12:55:24.017 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:24.015+0000 7f71b1611700 1 -- 172.21.15.71:0/1663845173 shutdown_connections 2024-02-20T12:55:24.018 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:24.015+0000 7f71b1611700 1 --2- 172.21.15.71:0/1663845173 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f71ac0f54b0 0x7f71ac0f58d0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:55:24.018 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:24.015+0000 7f71b1611700 1 -- 172.21.15.71:0/1663845173 >> 172.21.15.71:0/1663845173 conn(0x7f71ac0f0c10 msgr2=0x7f71ac0f3030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:55:24.018 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:24.015+0000 7f71b1611700 1 -- 172.21.15.71:0/1663845173 shutdown_connections 2024-02-20T12:55:24.018 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:24.015+0000 7f71b1611700 1 -- 172.21.15.71:0/1663845173 wait complete. 2024-02-20T12:55:24.018 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:24.016+0000 7f71b1611700 1 Processor -- start 2024-02-20T12:55:24.018 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:24.016+0000 7f71b1611700 1 -- start start 2024-02-20T12:55:24.018 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:24.016+0000 7f71b1611700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f71ac0f54b0 0x7f71ac1056f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:55:24.018 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:24.016+0000 7f71b1611700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71ac105be0 con 0x7f71ac0f54b0 2024-02-20T12:55:24.018 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:24.016+0000 7f71aaffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f71ac0f54b0 0x7f71ac1056f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:55:24.018 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:24.016+0000 7f71aaffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f71ac0f54b0 0x7f71ac1056f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.71:35642/0 (socket says 172.21.15.71:35642) 2024-02-20T12:55:24.018 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:24.016+0000 7f71aaffd700 1 -- 172.21.15.71:0/583493820 learned_addr learned my addr 172.21.15.71:0/583493820 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-02-20T12:55:24.018 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:24.016+0000 7f71aaffd700 1 -- 172.21.15.71:0/583493820 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71a0005040 con 0x7f71ac0f54b0 2024-02-20T12:55:24.018 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:24.017+0000 7f71aaffd700 1 --2- 172.21.15.71:0/583493820 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f71ac0f54b0 0x7f71ac1056f0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 rx=0x7f71a00089f0 tx=0x7f71a000b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:55:24.018 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:24.017+0000 7f71937fe700 1 -- 172.21.15.71:0/583493820 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f71a0009320 con 0x7f71ac0f54b0 2024-02-20T12:55:24.019 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:24.017+0000 7f71b1611700 1 -- 172.21.15.71:0/583493820 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71ac105de0 con 0x7f71ac0f54b0 2024-02-20T12:55:24.019 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:24.017+0000 7f71937fe700 1 -- 172.21.15.71:0/583493820 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(25 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f71a0013ec0 con 0x7f71ac0f54b0 2024-02-20T12:55:24.019 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:24.017+0000 7f71937fe700 1 -- 172.21.15.71:0/583493820 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f71a0014d30 con 0x7f71ac0f54b0 2024-02-20T12:55:24.019 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:24.017+0000 7f71b1611700 1 -- 172.21.15.71:0/583493820 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71ac106270 con 0x7f71ac0f54b0 2024-02-20T12:55:24.020 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:24.018+0000 7f71937fe700 1 -- 172.21.15.71:0/583493820 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 37151+0+0 (secure 0 0 0) 0x7f71a00095f0 con 0x7f71ac0f54b0 2024-02-20T12:55:24.020 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:24.018+0000 7f71b1611700 1 -- 172.21.15.71:0/583493820 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71ac045bb0 con 0x7f71ac0f54b0 2024-02-20T12:55:24.020 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:24.018+0000 7f71937fe700 1 --2- 172.21.15.71:0/583493820 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f7194031090 0x7f7194033550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:55:24.020 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:24.018+0000 7f71937fe700 1 -- 172.21.15.71:0/583493820 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1065+0+0 (secure 0 0 0) 0x7f71a00423e0 con 0x7f71ac0f54b0 2024-02-20T12:55:24.021 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:24.018+0000 7f71aa7fc700 1 --2- 172.21.15.71:0/583493820 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f7194031090 0x7f7194033550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:55:24.021 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:24.018+0000 7f71aa7fc700 1 --2- 172.21.15.71:0/583493820 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f7194031090 0x7f7194033550 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 rx=0x7f719c0096a0 tx=0x7f719c006b40).ready entity=mgr.14156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:55:24.023 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:24.023+0000 7f71937fe700 1 -- 172.21.15.71:0/583493820 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+153237 (secure 0 0 0) 0x7f71a00268c0 con 0x7f71ac0f54b0 2024-02-20T12:55:24.229 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:24.228+0000 7f71b1611700 1 -- 172.21.15.71:0/583493820 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f71ac107120 con 0x7f71ac0f54b0 2024-02-20T12:55:24.229 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:24.229+0000 7f71937fe700 1 -- 172.21.15.71:0/583493820 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7f71a001d380 con 0x7f71ac0f54b0 2024-02-20T12:55:24.230 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-20T12:55:24.230 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":1,"fsid":"680ee126-cfee-11ee-95bb-87774f69a715","modified":"2024-02-20T12:49:44.699059Z","created":"2024-02-20T12:49:44.699059Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi064","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:3300","nonce":0},{"type":"v1","addr":"172.21.15.64:6789","nonce":0}]},"addr":"172.21.15.64:6789/0","public_addr":"172.21.15.64:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-20T12:55:24.231 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:24.230+0000 7f71b1611700 1 -- 172.21.15.71:0/583493820 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f7194031090 msgr2=0x7f7194033550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:55:24.231 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:24.230+0000 7f71b1611700 1 --2- 172.21.15.71:0/583493820 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f7194031090 0x7f7194033550 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 rx=0x7f719c0096a0 tx=0x7f719c006b40).stop 2024-02-20T12:55:24.231 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:24.230+0000 7f71b1611700 1 -- 172.21.15.71:0/583493820 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f71ac0f54b0 msgr2=0x7f71ac1056f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:55:24.231 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:24.230+0000 7f71b1611700 1 --2- 172.21.15.71:0/583493820 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f71ac0f54b0 0x7f71ac1056f0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 rx=0x7f71a00089f0 tx=0x7f71a000b040).stop 2024-02-20T12:55:24.232 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:24.230+0000 7f71b1611700 1 -- 172.21.15.71:0/583493820 shutdown_connections 2024-02-20T12:55:24.232 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:24.230+0000 7f71b1611700 1 --2- 172.21.15.71:0/583493820 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f7194031090 0x7f7194033550 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:55:24.232 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:24.230+0000 7f71b1611700 1 --2- 172.21.15.71:0/583493820 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f71ac0f54b0 0x7f71ac1056f0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:55:24.232 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:24.230+0000 7f71b1611700 1 -- 172.21.15.71:0/583493820 >> 172.21.15.71:0/583493820 conn(0x7f71ac0f0c10 msgr2=0x7f71ac0f1690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:55:24.232 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:24.230+0000 7f71b1611700 1 -- 172.21.15.71:0/583493820 shutdown_connections 2024-02-20T12:55:24.232 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:24.230+0000 7f71b1611700 1 -- 172.21.15.71:0/583493820 wait complete. 2024-02-20T12:55:24.232 INFO:teuthology.orchestra.run.smithi071.stderr:dumped monmap epoch 1 2024-02-20T12:55:25.564 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:55:25 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:55:23.745646+0000 mgr.smithi064.vriyor (mgr.14156) 137 : cluster [DBG] pgmap v129: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:55:25.564 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:55:25 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:55:24.229674+0000 mon.smithi064 (mon.0) 177 : audit [DBG] from='client.? 172.21.15.71:0/583493820' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-20T12:55:25.906 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-20T12:55:25.906 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/daemon-base:latest-pacific shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 680ee126-cfee-11ee-95bb-87774f69a715 -- ceph mon dump -f json 2024-02-20T12:55:27.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:55:27 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:55:25.745930+0000 mgr.smithi064.vriyor (mgr.14156) 138 : cluster [DBG] pgmap v130: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:55:27.816 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:27.816+0000 7f8dad918700 1 -- 172.21.15.71:0/4172052261 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8da80f52d0 msgr2=0x7f8da80f56f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:55:27.817 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:27.816+0000 7f8dad918700 1 --2- 172.21.15.71:0/4172052261 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8da80f52d0 0x7f8da80f56f0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 rx=0x7f8d9c009fb0 tx=0x7f8d9c009b30).stop 2024-02-20T12:55:27.817 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:27.816+0000 7f8dad918700 1 -- 172.21.15.71:0/4172052261 shutdown_connections 2024-02-20T12:55:27.817 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:27.816+0000 7f8dad918700 1 --2- 172.21.15.71:0/4172052261 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8da80f52d0 0x7f8da80f56f0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:55:27.817 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:27.816+0000 7f8dad918700 1 -- 172.21.15.71:0/4172052261 >> 172.21.15.71:0/4172052261 conn(0x7f8da80f0a30 msgr2=0x7f8da80f2e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:55:27.817 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:27.816+0000 7f8dad918700 1 -- 172.21.15.71:0/4172052261 shutdown_connections 2024-02-20T12:55:27.817 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:27.816+0000 7f8dad918700 1 -- 172.21.15.71:0/4172052261 wait complete. 2024-02-20T12:55:27.818 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:27.817+0000 7f8dad918700 1 Processor -- start 2024-02-20T12:55:27.818 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:27.817+0000 7f8dad918700 1 -- start start 2024-02-20T12:55:27.819 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:27.817+0000 7f8dad918700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8da80f52d0 0x7f8da8109310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:55:27.819 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:27.817+0000 7f8dad918700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8da8060320 con 0x7f8da80f52d0 2024-02-20T12:55:27.819 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:27.817+0000 7f8dac916700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8da80f52d0 0x7f8da8109310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:55:27.819 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:27.817+0000 7f8dac916700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8da80f52d0 0x7f8da8109310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.71:50420/0 (socket says 172.21.15.71:50420) 2024-02-20T12:55:27.819 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:27.817+0000 7f8dac916700 1 -- 172.21.15.71:0/1761275882 learned_addr learned my addr 172.21.15.71:0/1761275882 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-02-20T12:55:27.819 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:27.818+0000 7f8dac916700 1 -- 172.21.15.71:0/1761275882 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d9c005040 con 0x7f8da80f52d0 2024-02-20T12:55:27.819 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:27.818+0000 7f8dac916700 1 --2- 172.21.15.71:0/1761275882 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8da80f52d0 0x7f8da8109310 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 rx=0x7f8d9c0089f0 tx=0x7f8d9c00b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:55:27.819 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:27.818+0000 7f8da57fa700 1 -- 172.21.15.71:0/1761275882 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8d9c009320 con 0x7f8da80f52d0 2024-02-20T12:55:27.819 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:27.818+0000 7f8da57fa700 1 -- 172.21.15.71:0/1761275882 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(25 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f8d9c013ec0 con 0x7f8da80f52d0 2024-02-20T12:55:27.819 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:27.818+0000 7f8dad918700 1 -- 172.21.15.71:0/1761275882 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8da8060520 con 0x7f8da80f52d0 2024-02-20T12:55:27.819 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:27.818+0000 7f8da57fa700 1 -- 172.21.15.71:0/1761275882 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8d9c014d30 con 0x7f8da80f52d0 2024-02-20T12:55:27.819 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:27.818+0000 7f8dad918700 1 -- 172.21.15.71:0/1761275882 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8da81099c0 con 0x7f8da80f52d0 2024-02-20T12:55:27.820 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:27.819+0000 7f8da57fa700 1 -- 172.21.15.71:0/1761275882 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 37151+0+0 (secure 0 0 0) 0x7f8d9c014420 con 0x7f8da80f52d0 2024-02-20T12:55:27.820 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:27.819+0000 7f8da57fa700 1 --2- 172.21.15.71:0/1761275882 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f8d98031090 0x7f8d98033550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:55:27.820 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:27.819+0000 7f8da57fa700 1 -- 172.21.15.71:0/1761275882 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1065+0+0 (secure 0 0 0) 0x7f8d9c0423e0 con 0x7f8da80f52d0 2024-02-20T12:55:27.820 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:27.819+0000 7f8dad918700 1 -- 172.21.15.71:0/1761275882 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8da8045bb0 con 0x7f8da80f52d0 2024-02-20T12:55:27.821 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:27.819+0000 7f8da7fff700 1 --2- 172.21.15.71:0/1761275882 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f8d98031090 0x7f8d98033550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:55:27.821 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:27.819+0000 7f8da7fff700 1 --2- 172.21.15.71:0/1761275882 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f8d98031090 0x7f8d98033550 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 rx=0x7f8d940096a0 tx=0x7f8d94006b40).ready entity=mgr.14156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:55:27.824 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:27.824+0000 7f8da57fa700 1 -- 172.21.15.71:0/1761275882 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+153237 (secure 0 0 0) 0x7f8d9c014750 con 0x7f8da80f52d0 2024-02-20T12:55:28.030 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:28.030+0000 7f8dad918700 1 -- 172.21.15.71:0/1761275882 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f8da810a1b0 con 0x7f8da80f52d0 2024-02-20T12:55:28.031 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:28.031+0000 7f8da57fa700 1 -- 172.21.15.71:0/1761275882 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7f8d9c01d7f0 con 0x7f8da80f52d0 2024-02-20T12:55:28.031 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-20T12:55:28.031 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":1,"fsid":"680ee126-cfee-11ee-95bb-87774f69a715","modified":"2024-02-20T12:49:44.699059Z","created":"2024-02-20T12:49:44.699059Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi064","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:3300","nonce":0},{"type":"v1","addr":"172.21.15.64:6789","nonce":0}]},"addr":"172.21.15.64:6789/0","public_addr":"172.21.15.64:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-20T12:55:28.033 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:28.032+0000 7f8dad918700 1 -- 172.21.15.71:0/1761275882 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f8d98031090 msgr2=0x7f8d98033550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:55:28.033 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:28.032+0000 7f8dad918700 1 --2- 172.21.15.71:0/1761275882 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f8d98031090 0x7f8d98033550 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 rx=0x7f8d940096a0 tx=0x7f8d94006b40).stop 2024-02-20T12:55:28.033 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:28.032+0000 7f8dad918700 1 -- 172.21.15.71:0/1761275882 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8da80f52d0 msgr2=0x7f8da8109310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:55:28.033 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:28.032+0000 7f8dad918700 1 --2- 172.21.15.71:0/1761275882 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8da80f52d0 0x7f8da8109310 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 rx=0x7f8d9c0089f0 tx=0x7f8d9c00b040).stop 2024-02-20T12:55:28.033 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:28.032+0000 7f8dad918700 1 -- 172.21.15.71:0/1761275882 shutdown_connections 2024-02-20T12:55:28.033 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:28.032+0000 7f8dad918700 1 --2- 172.21.15.71:0/1761275882 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f8d98031090 0x7f8d98033550 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:55:28.033 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:28.032+0000 7f8dad918700 1 --2- 172.21.15.71:0/1761275882 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8da80f52d0 0x7f8da8109310 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:55:28.033 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:28.032+0000 7f8dad918700 1 -- 172.21.15.71:0/1761275882 >> 172.21.15.71:0/1761275882 conn(0x7f8da80f0a30 msgr2=0x7f8da80f1550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:55:28.034 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:28.032+0000 7f8dad918700 1 -- 172.21.15.71:0/1761275882 shutdown_connections 2024-02-20T12:55:28.034 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:28.032+0000 7f8dad918700 1 -- 172.21.15.71:0/1761275882 wait complete. 2024-02-20T12:55:28.034 INFO:teuthology.orchestra.run.smithi071.stderr:dumped monmap epoch 1 2024-02-20T12:55:28.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:55:28 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:55:28.031344+0000 mon.smithi064 (mon.0) 178 : audit [DBG] from='client.? 172.21.15.71:0/1761275882' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-20T12:55:29.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:55:29 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:55:27.746194+0000 mgr.smithi064.vriyor (mgr.14156) 139 : cluster [DBG] pgmap v131: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:55:29.626 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-20T12:55:29.627 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/daemon-base:latest-pacific shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 680ee126-cfee-11ee-95bb-87774f69a715 -- ceph mon dump -f json 2024-02-20T12:55:31.442 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:31.440+0000 7f049d263700 1 -- 172.21.15.71:0/966919176 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f04980f54b0 msgr2=0x7f04980f58d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:55:31.442 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:31.440+0000 7f049d263700 1 --2- 172.21.15.71:0/966919176 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f04980f54b0 0x7f04980f58d0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 rx=0x7f0480009fb0 tx=0x7f0480009b30).stop 2024-02-20T12:55:31.442 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:31.440+0000 7f049d263700 1 -- 172.21.15.71:0/966919176 shutdown_connections 2024-02-20T12:55:31.442 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:31.440+0000 7f049d263700 1 --2- 172.21.15.71:0/966919176 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f04980f54b0 0x7f04980f58d0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:55:31.443 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:31.440+0000 7f049d263700 1 -- 172.21.15.71:0/966919176 >> 172.21.15.71:0/966919176 conn(0x7f04980f0c10 msgr2=0x7f04980f3030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:55:31.443 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:31.440+0000 7f049d263700 1 -- 172.21.15.71:0/966919176 shutdown_connections 2024-02-20T12:55:31.443 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:31.440+0000 7f049d263700 1 -- 172.21.15.71:0/966919176 wait complete. 2024-02-20T12:55:31.443 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:31.441+0000 7f049d263700 1 Processor -- start 2024-02-20T12:55:31.443 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:31.441+0000 7f049d263700 1 -- start start 2024-02-20T12:55:31.443 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:31.441+0000 7f049d263700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f04980f54b0 0x7f04981094a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:55:31.443 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:31.441+0000 7f049d263700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0498060320 con 0x7f04980f54b0 2024-02-20T12:55:31.443 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:31.441+0000 7f0496ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f04980f54b0 0x7f04981094a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:55:31.443 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:31.441+0000 7f0496ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f04980f54b0 0x7f04981094a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.71:50444/0 (socket says 172.21.15.71:50444) 2024-02-20T12:55:31.443 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:31.441+0000 7f0496ffd700 1 -- 172.21.15.71:0/2686974660 learned_addr learned my addr 172.21.15.71:0/2686974660 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-02-20T12:55:31.443 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:31.442+0000 7f0496ffd700 1 -- 172.21.15.71:0/2686974660 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0480005040 con 0x7f04980f54b0 2024-02-20T12:55:31.443 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:31.442+0000 7f0496ffd700 1 --2- 172.21.15.71:0/2686974660 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f04980f54b0 0x7f04981094a0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 rx=0x7f0480000c00 tx=0x7f048000b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:55:31.445 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:31.442+0000 7f048f7fe700 1 -- 172.21.15.71:0/2686974660 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f04800095f0 con 0x7f04980f54b0 2024-02-20T12:55:31.445 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:31.442+0000 7f048f7fe700 1 -- 172.21.15.71:0/2686974660 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(25 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f0480007e50 con 0x7f04980f54b0 2024-02-20T12:55:31.445 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:31.442+0000 7f049d263700 1 -- 172.21.15.71:0/2686974660 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0498060520 con 0x7f04980f54b0 2024-02-20T12:55:31.445 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:31.442+0000 7f048f7fe700 1 -- 172.21.15.71:0/2686974660 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0480014e70 con 0x7f04980f54b0 2024-02-20T12:55:31.445 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:31.442+0000 7f049d263700 1 -- 172.21.15.71:0/2686974660 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0498109b50 con 0x7f04980f54b0 2024-02-20T12:55:31.445 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:31.443+0000 7f048f7fe700 1 -- 172.21.15.71:0/2686974660 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 37151+0+0 (secure 0 0 0) 0x7f0480025070 con 0x7f04980f54b0 2024-02-20T12:55:31.446 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:31.443+0000 7f049d263700 1 -- 172.21.15.71:0/2686974660 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0498045bb0 con 0x7f04980f54b0 2024-02-20T12:55:31.446 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:31.443+0000 7f048f7fe700 1 --2- 172.21.15.71:0/2686974660 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f0484031090 0x7f0484033550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:55:31.446 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:31.443+0000 7f048f7fe700 1 -- 172.21.15.71:0/2686974660 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1065+0+0 (secure 0 0 0) 0x7f0480041d40 con 0x7f04980f54b0 2024-02-20T12:55:31.446 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:31.444+0000 7f04967fc700 1 --2- 172.21.15.71:0/2686974660 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f0484031090 0x7f0484033550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:55:31.446 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:31.444+0000 7f04967fc700 1 --2- 172.21.15.71:0/2686974660 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f0484031090 0x7f0484033550 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 rx=0x7f04880096a0 tx=0x7f0488006b40).ready entity=mgr.14156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:55:31.448 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:31.448+0000 7f048f7fe700 1 -- 172.21.15.71:0/2686974660 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+153237 (secure 0 0 0) 0x7f048000f4e0 con 0x7f04980f54b0 2024-02-20T12:55:31.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:55:31 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:55:29.746493+0000 mgr.smithi064.vriyor (mgr.14156) 140 : cluster [DBG] pgmap v132: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:55:31.656 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:31.656+0000 7f049d263700 1 -- 172.21.15.71:0/2686974660 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f049810a330 con 0x7f04980f54b0 2024-02-20T12:55:31.657 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:31.656+0000 7f048f7fe700 1 -- 172.21.15.71:0/2686974660 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7f048000f300 con 0x7f04980f54b0 2024-02-20T12:55:31.657 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-20T12:55:31.657 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":1,"fsid":"680ee126-cfee-11ee-95bb-87774f69a715","modified":"2024-02-20T12:49:44.699059Z","created":"2024-02-20T12:49:44.699059Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi064","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:3300","nonce":0},{"type":"v1","addr":"172.21.15.64:6789","nonce":0}]},"addr":"172.21.15.64:6789/0","public_addr":"172.21.15.64:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-20T12:55:31.658 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:31.658+0000 7f049d263700 1 -- 172.21.15.71:0/2686974660 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f0484031090 msgr2=0x7f0484033550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:55:31.658 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:31.658+0000 7f049d263700 1 --2- 172.21.15.71:0/2686974660 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f0484031090 0x7f0484033550 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 rx=0x7f04880096a0 tx=0x7f0488006b40).stop 2024-02-20T12:55:31.659 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:31.658+0000 7f049d263700 1 -- 172.21.15.71:0/2686974660 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f04980f54b0 msgr2=0x7f04981094a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:55:31.659 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:31.658+0000 7f049d263700 1 --2- 172.21.15.71:0/2686974660 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f04980f54b0 0x7f04981094a0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 rx=0x7f0480000c00 tx=0x7f048000b040).stop 2024-02-20T12:55:31.659 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:31.658+0000 7f049d263700 1 -- 172.21.15.71:0/2686974660 shutdown_connections 2024-02-20T12:55:31.659 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:31.658+0000 7f049d263700 1 --2- 172.21.15.71:0/2686974660 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f0484031090 0x7f0484033550 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:55:31.659 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:31.658+0000 7f049d263700 1 --2- 172.21.15.71:0/2686974660 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f04980f54b0 0x7f04981094a0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:55:31.659 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:31.658+0000 7f049d263700 1 -- 172.21.15.71:0/2686974660 >> 172.21.15.71:0/2686974660 conn(0x7f04980f0c10 msgr2=0x7f04980f1670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:55:31.659 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:31.658+0000 7f049d263700 1 -- 172.21.15.71:0/2686974660 shutdown_connections 2024-02-20T12:55:31.659 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:31.658+0000 7f049d263700 1 -- 172.21.15.71:0/2686974660 wait complete. 2024-02-20T12:55:31.659 INFO:teuthology.orchestra.run.smithi071.stderr:dumped monmap epoch 1 2024-02-20T12:55:32.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:55:32 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:55:31.657278+0000 mon.smithi064 (mon.0) 179 : audit [DBG] from='client.? 172.21.15.71:0/2686974660' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-20T12:55:33.218 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-20T12:55:33.218 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/daemon-base:latest-pacific shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 680ee126-cfee-11ee-95bb-87774f69a715 -- ceph mon dump -f json 2024-02-20T12:55:33.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:55:33 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:55:31.746842+0000 mgr.smithi064.vriyor (mgr.14156) 141 : cluster [DBG] pgmap v133: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:55:35.001 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:35.000+0000 7fafb7048700 1 -- 172.21.15.71:0/42013634 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fafb00f5480 msgr2=0x7fafb00f58a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:55:35.001 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:35.000+0000 7fafb7048700 1 --2- 172.21.15.71:0/42013634 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fafb00f5480 0x7fafb00f58a0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 rx=0x7fafa4009fb0 tx=0x7fafa4009b30).stop 2024-02-20T12:55:35.002 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:35.000+0000 7fafb7048700 1 -- 172.21.15.71:0/42013634 shutdown_connections 2024-02-20T12:55:35.002 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:35.000+0000 7fafb7048700 1 --2- 172.21.15.71:0/42013634 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fafb00f5480 0x7fafb00f58a0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:55:35.002 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:35.000+0000 7fafb7048700 1 -- 172.21.15.71:0/42013634 >> 172.21.15.71:0/42013634 conn(0x7fafb00f0be0 msgr2=0x7fafb00f3000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:55:35.002 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:35.001+0000 7fafb7048700 1 -- 172.21.15.71:0/42013634 shutdown_connections 2024-02-20T12:55:35.002 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:35.001+0000 7fafb7048700 1 -- 172.21.15.71:0/42013634 wait complete. 2024-02-20T12:55:35.002 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:35.001+0000 7fafb7048700 1 Processor -- start 2024-02-20T12:55:35.002 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:35.001+0000 7fafb7048700 1 -- start start 2024-02-20T12:55:35.004 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:35.001+0000 7fafb7048700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fafb00f5480 0x7fafb01067f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:55:35.004 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:35.001+0000 7fafb7048700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafb0106ce0 con 0x7fafb00f5480 2024-02-20T12:55:35.004 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:35.002+0000 7fafb4de4700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fafb00f5480 0x7fafb01067f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:55:35.004 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:35.002+0000 7fafb4de4700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fafb00f5480 0x7fafb01067f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.71:50462/0 (socket says 172.21.15.71:50462) 2024-02-20T12:55:35.004 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:35.002+0000 7fafb4de4700 1 -- 172.21.15.71:0/3553454730 learned_addr learned my addr 172.21.15.71:0/3553454730 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-02-20T12:55:35.004 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:35.002+0000 7fafb4de4700 1 -- 172.21.15.71:0/3553454730 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafa4005040 con 0x7fafb00f5480 2024-02-20T12:55:35.004 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:35.002+0000 7fafb4de4700 1 --2- 172.21.15.71:0/3553454730 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fafb00f5480 0x7fafb01067f0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 rx=0x7fafa40089f0 tx=0x7fafa400b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:55:35.004 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:35.002+0000 7fafad7fa700 1 -- 172.21.15.71:0/3553454730 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fafa4009320 con 0x7fafb00f5480 2024-02-20T12:55:35.004 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:35.002+0000 7fafb7048700 1 -- 172.21.15.71:0/3553454730 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fafb0106ee0 con 0x7fafb00f5480 2024-02-20T12:55:35.004 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:35.002+0000 7fafad7fa700 1 -- 172.21.15.71:0/3553454730 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(25 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7fafa4013ec0 con 0x7fafb00f5480 2024-02-20T12:55:35.005 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:35.002+0000 7fafad7fa700 1 -- 172.21.15.71:0/3553454730 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fafa4014d30 con 0x7fafb00f5480 2024-02-20T12:55:35.005 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:35.002+0000 7fafb7048700 1 -- 172.21.15.71:0/3553454730 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fafb0107370 con 0x7fafb00f5480 2024-02-20T12:55:35.005 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:35.003+0000 7fafad7fa700 1 -- 172.21.15.71:0/3553454730 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 37151+0+0 (secure 0 0 0) 0x7fafa40263a0 con 0x7fafb00f5480 2024-02-20T12:55:35.005 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:35.003+0000 7fafad7fa700 1 --2- 172.21.15.71:0/3553454730 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fafa0031090 0x7fafa0033550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:55:35.005 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:35.003+0000 7fafad7fa700 1 -- 172.21.15.71:0/3553454730 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1065+0+0 (secure 0 0 0) 0x7fafa4042420 con 0x7fafb00f5480 2024-02-20T12:55:35.005 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:35.003+0000 7fafb7048700 1 -- 172.21.15.71:0/3553454730 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fafb00591e0 con 0x7fafb00f5480 2024-02-20T12:55:35.005 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:35.004+0000 7fafaffff700 1 --2- 172.21.15.71:0/3553454730 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fafa0031090 0x7fafa0033550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:55:35.006 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:35.004+0000 7fafaffff700 1 --2- 172.21.15.71:0/3553454730 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fafa0031090 0x7fafa0033550 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 rx=0x7faf9c0096a0 tx=0x7faf9c006b40).ready entity=mgr.14156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:55:35.009 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:35.008+0000 7fafad7fa700 1 -- 172.21.15.71:0/3553454730 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+153237 (secure 0 0 0) 0x7fafa40266d0 con 0x7fafb00f5480 2024-02-20T12:55:35.214 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:35.213+0000 7fafb7048700 1 -- 172.21.15.71:0/3553454730 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fafb00fc0a0 con 0x7fafb00f5480 2024-02-20T12:55:35.214 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:35.214+0000 7fafad7fa700 1 -- 172.21.15.71:0/3553454730 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7fafa402b020 con 0x7fafb00f5480 2024-02-20T12:55:35.215 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-20T12:55:35.215 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":1,"fsid":"680ee126-cfee-11ee-95bb-87774f69a715","modified":"2024-02-20T12:49:44.699059Z","created":"2024-02-20T12:49:44.699059Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi064","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:3300","nonce":0},{"type":"v1","addr":"172.21.15.64:6789","nonce":0}]},"addr":"172.21.15.64:6789/0","public_addr":"172.21.15.64:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-20T12:55:35.216 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:35.216+0000 7fafb7048700 1 -- 172.21.15.71:0/3553454730 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fafa0031090 msgr2=0x7fafa0033550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:55:35.216 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:35.216+0000 7fafb7048700 1 --2- 172.21.15.71:0/3553454730 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fafa0031090 0x7fafa0033550 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 rx=0x7faf9c0096a0 tx=0x7faf9c006b40).stop 2024-02-20T12:55:35.217 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:35.216+0000 7fafb7048700 1 -- 172.21.15.71:0/3553454730 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fafb00f5480 msgr2=0x7fafb01067f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:55:35.217 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:35.216+0000 7fafb7048700 1 --2- 172.21.15.71:0/3553454730 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fafb00f5480 0x7fafb01067f0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 rx=0x7fafa40089f0 tx=0x7fafa400b040).stop 2024-02-20T12:55:35.217 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:35.216+0000 7fafb7048700 1 -- 172.21.15.71:0/3553454730 shutdown_connections 2024-02-20T12:55:35.217 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:35.216+0000 7fafb7048700 1 --2- 172.21.15.71:0/3553454730 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fafa0031090 0x7fafa0033550 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:55:35.217 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:35.216+0000 7fafb7048700 1 --2- 172.21.15.71:0/3553454730 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fafb00f5480 0x7fafb01067f0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:55:35.217 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:35.216+0000 7fafb7048700 1 -- 172.21.15.71:0/3553454730 >> 172.21.15.71:0/3553454730 conn(0x7fafb00f0be0 msgr2=0x7fafb00f16d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:55:35.217 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:35.216+0000 7fafb7048700 1 -- 172.21.15.71:0/3553454730 shutdown_connections 2024-02-20T12:55:35.217 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:35.216+0000 7fafb7048700 1 -- 172.21.15.71:0/3553454730 wait complete. 2024-02-20T12:55:35.217 INFO:teuthology.orchestra.run.smithi071.stderr:dumped monmap epoch 1 2024-02-20T12:55:35.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:55:35 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:55:33.747146+0000 mgr.smithi064.vriyor (mgr.14156) 142 : cluster [DBG] pgmap v134: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:55:36.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:55:36 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:55:35.215060+0000 mon.smithi064 (mon.0) 180 : audit [DBG] from='client.? 172.21.15.71:0/3553454730' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-20T12:55:36.863 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-20T12:55:36.863 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/daemon-base:latest-pacific shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 680ee126-cfee-11ee-95bb-87774f69a715 -- ceph mon dump -f json 2024-02-20T12:55:37.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:55:37 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:55:35.747496+0000 mgr.smithi064.vriyor (mgr.14156) 143 : cluster [DBG] pgmap v135: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:55:38.697 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:38.692+0000 7fe6b8527700 1 -- 172.21.15.71:0/2459463172 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe6b00f6590 msgr2=0x7fe6b00f69b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:55:38.697 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:38.692+0000 7fe6b8527700 1 --2- 172.21.15.71:0/2459463172 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe6b00f6590 0x7fe6b00f69b0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 rx=0x7fe6a0004760 tx=0x7fe6a0009ad0).stop 2024-02-20T12:55:38.697 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:38.692+0000 7fe6b8527700 1 -- 172.21.15.71:0/2459463172 shutdown_connections 2024-02-20T12:55:38.697 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:38.692+0000 7fe6b8527700 1 --2- 172.21.15.71:0/2459463172 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe6b00f6590 0x7fe6b00f69b0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:55:38.697 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:38.692+0000 7fe6b8527700 1 -- 172.21.15.71:0/2459463172 >> 172.21.15.71:0/2459463172 conn(0x7fe6b00f1cd0 msgr2=0x7fe6b00f4110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:55:38.697 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:38.692+0000 7fe6b8527700 1 -- 172.21.15.71:0/2459463172 shutdown_connections 2024-02-20T12:55:38.697 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:38.692+0000 7fe6b8527700 1 -- 172.21.15.71:0/2459463172 wait complete. 2024-02-20T12:55:38.697 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:38.693+0000 7fe6b8527700 1 Processor -- start 2024-02-20T12:55:38.698 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:38.693+0000 7fe6b8527700 1 -- start start 2024-02-20T12:55:38.698 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:38.693+0000 7fe6b8527700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe6b00f6590 0x7fe6b006ba80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:55:38.698 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:38.693+0000 7fe6b8527700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6b0069f00 con 0x7fe6b00f6590 2024-02-20T12:55:38.698 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:38.693+0000 7fe6b62c3700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe6b00f6590 0x7fe6b006ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:55:38.698 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:38.693+0000 7fe6b62c3700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe6b00f6590 0x7fe6b006ba80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.71:55948/0 (socket says 172.21.15.71:55948) 2024-02-20T12:55:38.698 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:38.693+0000 7fe6b62c3700 1 -- 172.21.15.71:0/3590382683 learned_addr learned my addr 172.21.15.71:0/3590382683 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-02-20T12:55:38.698 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:38.693+0000 7fe6b62c3700 1 -- 172.21.15.71:0/3590382683 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6a0009d70 con 0x7fe6b00f6590 2024-02-20T12:55:38.698 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:38.694+0000 7fe6b62c3700 1 --2- 172.21.15.71:0/3590382683 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe6b00f6590 0x7fe6b006ba80 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 rx=0x7fe6a0004170 tx=0x7fe6a0009ad0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:55:38.698 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:38.694+0000 7fe6a6ffd700 1 -- 172.21.15.71:0/3590382683 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe6a000e040 con 0x7fe6b00f6590 2024-02-20T12:55:38.698 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:38.694+0000 7fe6b8527700 1 -- 172.21.15.71:0/3590382683 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6b006a0a0 con 0x7fe6b00f6590 2024-02-20T12:55:38.698 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:38.694+0000 7fe6a6ffd700 1 -- 172.21.15.71:0/3590382683 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(25 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7fe6a0008ef0 con 0x7fe6b00f6590 2024-02-20T12:55:38.698 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:38.694+0000 7fe6a6ffd700 1 -- 172.21.15.71:0/3590382683 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe6a0014c90 con 0x7fe6b00f6590 2024-02-20T12:55:38.698 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:38.694+0000 7fe6b8527700 1 -- 172.21.15.71:0/3590382683 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6b006a530 con 0x7fe6b00f6590 2024-02-20T12:55:38.699 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:38.694+0000 7fe6a6ffd700 1 -- 172.21.15.71:0/3590382683 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 37151+0+0 (secure 0 0 0) 0x7fe6a00283f0 con 0x7fe6b00f6590 2024-02-20T12:55:38.699 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:38.695+0000 7fe6a6ffd700 1 --2- 172.21.15.71:0/3590382683 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fe69c031090 0x7fe69c033550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:55:38.699 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:38.695+0000 7fe6a6ffd700 1 -- 172.21.15.71:0/3590382683 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1065+0+0 (secure 0 0 0) 0x7fe6a0042220 con 0x7fe6b00f6590 2024-02-20T12:55:38.699 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:38.695+0000 7fe6b8527700 1 -- 172.21.15.71:0/3590382683 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6b00591e0 con 0x7fe6b00f6590 2024-02-20T12:55:38.699 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:38.695+0000 7fe6b5ac2700 1 --2- 172.21.15.71:0/3590382683 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fe69c031090 0x7fe69c033550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:55:38.699 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:38.695+0000 7fe6b5ac2700 1 --2- 172.21.15.71:0/3590382683 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fe69c031090 0x7fe69c033550 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 rx=0x7fe6ac0096a0 tx=0x7fe6ac006b40).ready entity=mgr.14156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:55:38.701 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:38.700+0000 7fe6a6ffd700 1 -- 172.21.15.71:0/3590382683 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+153237 (secure 0 0 0) 0x7fe6a00268f0 con 0x7fe6b00f6590 2024-02-20T12:55:38.907 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:38.905+0000 7fe6b8527700 1 -- 172.21.15.71:0/3590382683 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fe6b006ab90 con 0x7fe6b00f6590 2024-02-20T12:55:38.907 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:38.906+0000 7fe6a6ffd700 1 -- 172.21.15.71:0/3590382683 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7fe6a0026300 con 0x7fe6b00f6590 2024-02-20T12:55:38.908 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-20T12:55:38.908 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":1,"fsid":"680ee126-cfee-11ee-95bb-87774f69a715","modified":"2024-02-20T12:49:44.699059Z","created":"2024-02-20T12:49:44.699059Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi064","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:3300","nonce":0},{"type":"v1","addr":"172.21.15.64:6789","nonce":0}]},"addr":"172.21.15.64:6789/0","public_addr":"172.21.15.64:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-20T12:55:38.909 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:38.908+0000 7fe6b8527700 1 -- 172.21.15.71:0/3590382683 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fe69c031090 msgr2=0x7fe69c033550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:55:38.909 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:38.908+0000 7fe6b8527700 1 --2- 172.21.15.71:0/3590382683 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fe69c031090 0x7fe69c033550 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 rx=0x7fe6ac0096a0 tx=0x7fe6ac006b40).stop 2024-02-20T12:55:38.910 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:38.908+0000 7fe6b8527700 1 -- 172.21.15.71:0/3590382683 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe6b00f6590 msgr2=0x7fe6b006ba80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:55:38.910 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:38.908+0000 7fe6b8527700 1 --2- 172.21.15.71:0/3590382683 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe6b00f6590 0x7fe6b006ba80 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 rx=0x7fe6a0004170 tx=0x7fe6a0009ad0).stop 2024-02-20T12:55:38.910 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:38.908+0000 7fe6b8527700 1 -- 172.21.15.71:0/3590382683 shutdown_connections 2024-02-20T12:55:38.910 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:38.908+0000 7fe6b8527700 1 --2- 172.21.15.71:0/3590382683 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fe69c031090 0x7fe69c033550 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:55:38.910 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:38.908+0000 7fe6b8527700 1 --2- 172.21.15.71:0/3590382683 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe6b00f6590 0x7fe6b006ba80 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:55:38.910 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:38.908+0000 7fe6b8527700 1 -- 172.21.15.71:0/3590382683 >> 172.21.15.71:0/3590382683 conn(0x7fe6b00f1cd0 msgr2=0x7fe6b00f27c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:55:38.910 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:38.908+0000 7fe6b8527700 1 -- 172.21.15.71:0/3590382683 shutdown_connections 2024-02-20T12:55:38.910 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:38.908+0000 7fe6b8527700 1 -- 172.21.15.71:0/3590382683 wait complete. 2024-02-20T12:55:38.910 INFO:teuthology.orchestra.run.smithi071.stderr:dumped monmap epoch 1 2024-02-20T12:55:39.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:55:39 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:55:37.747789+0000 mgr.smithi064.vriyor (mgr.14156) 144 : cluster [DBG] pgmap v136: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:55:39.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:55:39 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:55:38.908013+0000 mon.smithi064 (mon.0) 181 : audit [DBG] from='client.? 172.21.15.71:0/3590382683' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-20T12:55:40.595 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-20T12:55:40.596 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/daemon-base:latest-pacific shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 680ee126-cfee-11ee-95bb-87774f69a715 -- ceph mon dump -f json 2024-02-20T12:55:41.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:55:41 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:55:39.748153+0000 mgr.smithi064.vriyor (mgr.14156) 145 : cluster [DBG] pgmap v137: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:55:42.461 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:42.448+0000 7fdd5a66f700 1 -- 172.21.15.71:0/235845692 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdd54060360 msgr2=0x7fdd540f9190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:55:42.461 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:42.448+0000 7fdd5a66f700 1 --2- 172.21.15.71:0/235845692 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdd54060360 0x7fdd540f9190 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 rx=0x7fdd3c009fb0 tx=0x7fdd3c009b30).stop 2024-02-20T12:55:42.461 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:42.448+0000 7fdd5a66f700 1 -- 172.21.15.71:0/235845692 shutdown_connections 2024-02-20T12:55:42.461 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:42.448+0000 7fdd5a66f700 1 --2- 172.21.15.71:0/235845692 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdd54060360 0x7fdd540f9190 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:55:42.461 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:42.448+0000 7fdd5a66f700 1 -- 172.21.15.71:0/235845692 >> 172.21.15.71:0/235845692 conn(0x7fdd540f0c10 msgr2=0x7fdd540f3030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:55:42.461 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:42.448+0000 7fdd5a66f700 1 -- 172.21.15.71:0/235845692 shutdown_connections 2024-02-20T12:55:42.462 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:42.448+0000 7fdd5a66f700 1 -- 172.21.15.71:0/235845692 wait complete. 2024-02-20T12:55:42.462 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:42.448+0000 7fdd5a66f700 1 Processor -- start 2024-02-20T12:55:42.462 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:42.449+0000 7fdd5a66f700 1 -- start start 2024-02-20T12:55:42.462 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:42.449+0000 7fdd5a66f700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdd54060360 0x7fdd54106770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:55:42.462 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:42.449+0000 7fdd5a66f700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd54106c60 con 0x7fdd54060360 2024-02-20T12:55:42.462 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:42.449+0000 7fdd53fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdd54060360 0x7fdd54106770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:55:42.462 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:42.449+0000 7fdd53fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdd54060360 0x7fdd54106770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.71:55952/0 (socket says 172.21.15.71:55952) 2024-02-20T12:55:42.462 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:42.449+0000 7fdd53fff700 1 -- 172.21.15.71:0/1150873752 learned_addr learned my addr 172.21.15.71:0/1150873752 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-02-20T12:55:42.462 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:42.449+0000 7fdd53fff700 1 -- 172.21.15.71:0/1150873752 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd3c005040 con 0x7fdd54060360 2024-02-20T12:55:42.462 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:42.449+0000 7fdd53fff700 1 --2- 172.21.15.71:0/1150873752 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdd54060360 0x7fdd54106770 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 rx=0x7fdd3c009f80 tx=0x7fdd3c00b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:55:42.462 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:42.449+0000 7fdd50ff9700 1 -- 172.21.15.71:0/1150873752 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fdd3c009320 con 0x7fdd54060360 2024-02-20T12:55:42.462 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:42.450+0000 7fdd5a66f700 1 -- 172.21.15.71:0/1150873752 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd54106e60 con 0x7fdd54060360 2024-02-20T12:55:42.462 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:42.450+0000 7fdd50ff9700 1 -- 172.21.15.71:0/1150873752 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(25 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7fdd3c013680 con 0x7fdd54060360 2024-02-20T12:55:42.462 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:42.450+0000 7fdd50ff9700 1 -- 172.21.15.71:0/1150873752 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fdd3c01eda0 con 0x7fdd54060360 2024-02-20T12:55:42.462 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:42.450+0000 7fdd5a66f700 1 -- 172.21.15.71:0/1150873752 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd541072f0 con 0x7fdd54060360 2024-02-20T12:55:42.463 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:42.450+0000 7fdd50ff9700 1 -- 172.21.15.71:0/1150873752 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 37151+0+0 (secure 0 0 0) 0x7fdd3c01e420 con 0x7fdd54060360 2024-02-20T12:55:42.463 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:42.450+0000 7fdd5a66f700 1 -- 172.21.15.71:0/1150873752 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd54045bb0 con 0x7fdd54060360 2024-02-20T12:55:42.463 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:42.451+0000 7fdd50ff9700 1 --2- 172.21.15.71:0/1150873752 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fdd40031090 0x7fdd40033550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:55:42.463 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:42.451+0000 7fdd50ff9700 1 -- 172.21.15.71:0/1150873752 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1065+0+0 (secure 0 0 0) 0x7fdd3c042430 con 0x7fdd54060360 2024-02-20T12:55:42.463 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:42.451+0000 7fdd537fe700 1 --2- 172.21.15.71:0/1150873752 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fdd40031090 0x7fdd40033550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:55:42.463 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:42.451+0000 7fdd537fe700 1 --2- 172.21.15.71:0/1150873752 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fdd40031090 0x7fdd40033550 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 rx=0x7fdd440096a0 tx=0x7fdd44006b40).ready entity=mgr.14156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:55:42.463 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:42.455+0000 7fdd50ff9700 1 -- 172.21.15.71:0/1150873752 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+153237 (secure 0 0 0) 0x7fdd3c01d380 con 0x7fdd54060360 2024-02-20T12:55:42.664 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:42.663+0000 7fdd5a66f700 1 -- 172.21.15.71:0/1150873752 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fdd54108140 con 0x7fdd54060360 2024-02-20T12:55:42.665 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:42.663+0000 7fdd50ff9700 1 -- 172.21.15.71:0/1150873752 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7fdd3c01cb10 con 0x7fdd54060360 2024-02-20T12:55:42.665 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-20T12:55:42.666 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":1,"fsid":"680ee126-cfee-11ee-95bb-87774f69a715","modified":"2024-02-20T12:49:44.699059Z","created":"2024-02-20T12:49:44.699059Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi064","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:3300","nonce":0},{"type":"v1","addr":"172.21.15.64:6789","nonce":0}]},"addr":"172.21.15.64:6789/0","public_addr":"172.21.15.64:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-20T12:55:42.667 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:42.665+0000 7fdd5a66f700 1 -- 172.21.15.71:0/1150873752 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fdd40031090 msgr2=0x7fdd40033550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:55:42.667 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:42.665+0000 7fdd5a66f700 1 --2- 172.21.15.71:0/1150873752 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fdd40031090 0x7fdd40033550 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 rx=0x7fdd440096a0 tx=0x7fdd44006b40).stop 2024-02-20T12:55:42.667 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:42.665+0000 7fdd5a66f700 1 -- 172.21.15.71:0/1150873752 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdd54060360 msgr2=0x7fdd54106770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:55:42.668 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:42.665+0000 7fdd5a66f700 1 --2- 172.21.15.71:0/1150873752 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdd54060360 0x7fdd54106770 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 rx=0x7fdd3c009f80 tx=0x7fdd3c00b040).stop 2024-02-20T12:55:42.668 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:42.665+0000 7fdd5a66f700 1 -- 172.21.15.71:0/1150873752 shutdown_connections 2024-02-20T12:55:42.668 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:42.665+0000 7fdd5a66f700 1 --2- 172.21.15.71:0/1150873752 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fdd40031090 0x7fdd40033550 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:55:42.668 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:42.665+0000 7fdd5a66f700 1 --2- 172.21.15.71:0/1150873752 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdd54060360 0x7fdd54106770 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:55:42.668 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:42.665+0000 7fdd5a66f700 1 -- 172.21.15.71:0/1150873752 >> 172.21.15.71:0/1150873752 conn(0x7fdd540f0c10 msgr2=0x7fdd540f1690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:55:42.668 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:42.665+0000 7fdd5a66f700 1 -- 172.21.15.71:0/1150873752 shutdown_connections 2024-02-20T12:55:42.668 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:42.665+0000 7fdd5a66f700 1 -- 172.21.15.71:0/1150873752 wait complete. 2024-02-20T12:55:42.668 INFO:teuthology.orchestra.run.smithi071.stderr:dumped monmap epoch 1 2024-02-20T12:55:43.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:55:43 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:55:41.748412+0000 mgr.smithi064.vriyor (mgr.14156) 146 : cluster [DBG] pgmap v138: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:55:43.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:55:43 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:55:42.665102+0000 mon.smithi064 (mon.0) 182 : audit [DBG] from='client.? 172.21.15.71:0/1150873752' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-20T12:55:44.274 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-20T12:55:44.274 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/daemon-base:latest-pacific shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 680ee126-cfee-11ee-95bb-87774f69a715 -- ceph mon dump -f json 2024-02-20T12:55:45.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:55:45 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:55:43.748633+0000 mgr.smithi064.vriyor (mgr.14156) 147 : cluster [DBG] pgmap v139: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:55:46.163 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:46.143+0000 7f4bfce1f700 1 -- 172.21.15.71:0/2472309019 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4bf80f52d0 msgr2=0x7f4bf80f56f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:55:46.163 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:46.143+0000 7f4bfce1f700 1 --2- 172.21.15.71:0/2472309019 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4bf80f52d0 0x7f4bf80f56f0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 rx=0x7f4be8009fb0 tx=0x7f4be8009b30).stop 2024-02-20T12:55:46.163 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:46.143+0000 7f4bfce1f700 1 -- 172.21.15.71:0/2472309019 shutdown_connections 2024-02-20T12:55:46.164 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:46.143+0000 7f4bfce1f700 1 --2- 172.21.15.71:0/2472309019 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4bf80f52d0 0x7f4bf80f56f0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:55:46.164 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:46.143+0000 7f4bfce1f700 1 -- 172.21.15.71:0/2472309019 >> 172.21.15.71:0/2472309019 conn(0x7f4bf80f0a30 msgr2=0x7f4bf80f2e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:55:46.164 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:46.144+0000 7f4bfce1f700 1 -- 172.21.15.71:0/2472309019 shutdown_connections 2024-02-20T12:55:46.164 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:46.144+0000 7f4bfce1f700 1 -- 172.21.15.71:0/2472309019 wait complete. 2024-02-20T12:55:46.164 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:46.144+0000 7f4bfce1f700 1 Processor -- start 2024-02-20T12:55:46.164 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:46.144+0000 7f4bfce1f700 1 -- start start 2024-02-20T12:55:46.164 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:46.144+0000 7f4bfce1f700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4bf80f52d0 0x7f4bf810b730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:55:46.164 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:46.144+0000 7f4bfce1f700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bf8107120 con 0x7f4bf80f52d0 2024-02-20T12:55:46.165 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:46.145+0000 7f4bf77fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4bf80f52d0 0x7f4bf810b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:55:46.165 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:46.145+0000 7f4bf77fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4bf80f52d0 0x7f4bf810b730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.71:55980/0 (socket says 172.21.15.71:55980) 2024-02-20T12:55:46.165 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:46.145+0000 7f4bf77fe700 1 -- 172.21.15.71:0/2863775952 learned_addr learned my addr 172.21.15.71:0/2863775952 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-02-20T12:55:46.165 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:46.145+0000 7f4bf77fe700 1 -- 172.21.15.71:0/2863775952 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4be8005040 con 0x7f4bf80f52d0 2024-02-20T12:55:46.165 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:46.145+0000 7f4bf77fe700 1 --2- 172.21.15.71:0/2863775952 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4bf80f52d0 0x7f4bf810b730 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 rx=0x7f4be80089f0 tx=0x7f4be800b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:55:46.165 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:46.145+0000 7f4bdffff700 1 -- 172.21.15.71:0/2863775952 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4be8009320 con 0x7f4bf80f52d0 2024-02-20T12:55:46.165 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:46.145+0000 7f4bfce1f700 1 -- 172.21.15.71:0/2863775952 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4bf8060300 con 0x7f4bf80f52d0 2024-02-20T12:55:46.166 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:46.145+0000 7f4bdffff700 1 -- 172.21.15.71:0/2863775952 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(25 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f4be8013ec0 con 0x7f4bf80f52d0 2024-02-20T12:55:46.166 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:46.145+0000 7f4bdffff700 1 -- 172.21.15.71:0/2863775952 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4be8014d30 con 0x7f4bf80f52d0 2024-02-20T12:55:46.166 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:46.145+0000 7f4bfce1f700 1 -- 172.21.15.71:0/2863775952 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4bf810bc20 con 0x7f4bf80f52d0 2024-02-20T12:55:46.166 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:46.146+0000 7f4bdffff700 1 -- 172.21.15.71:0/2863775952 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 37151+0+0 (secure 0 0 0) 0x7f4be8014420 con 0x7f4bf80f52d0 2024-02-20T12:55:46.166 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:46.146+0000 7f4bfce1f700 1 -- 172.21.15.71:0/2863775952 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4bf8045bb0 con 0x7f4bf80f52d0 2024-02-20T12:55:46.166 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:46.146+0000 7f4bdffff700 1 --2- 172.21.15.71:0/2863775952 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f4be0031090 0x7f4be0033550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:55:46.166 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:46.146+0000 7f4bdffff700 1 -- 172.21.15.71:0/2863775952 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1065+0+0 (secure 0 0 0) 0x7f4be80423e0 con 0x7f4bf80f52d0 2024-02-20T12:55:46.167 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:46.146+0000 7f4bf6ffd700 1 --2- 172.21.15.71:0/2863775952 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f4be0031090 0x7f4be0033550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:55:46.167 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:46.147+0000 7f4bf6ffd700 1 --2- 172.21.15.71:0/2863775952 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f4be0031090 0x7f4be0033550 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 rx=0x7f4bec0096a0 tx=0x7f4bec006b40).ready entity=mgr.14156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:55:46.167 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:46.151+0000 7f4bdffff700 1 -- 172.21.15.71:0/2863775952 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+153237 (secure 0 0 0) 0x7f4be80263a0 con 0x7f4bf80f52d0 2024-02-20T12:55:46.360 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:46.359+0000 7f4bfce1f700 1 -- 172.21.15.71:0/2863775952 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f4bf810c440 con 0x7f4bf80f52d0 2024-02-20T12:55:46.361 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:46.360+0000 7f4bdffff700 1 -- 172.21.15.71:0/2863775952 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7f4be801d7f0 con 0x7f4bf80f52d0 2024-02-20T12:55:46.361 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-20T12:55:46.361 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":1,"fsid":"680ee126-cfee-11ee-95bb-87774f69a715","modified":"2024-02-20T12:49:44.699059Z","created":"2024-02-20T12:49:44.699059Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi064","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:3300","nonce":0},{"type":"v1","addr":"172.21.15.64:6789","nonce":0}]},"addr":"172.21.15.64:6789/0","public_addr":"172.21.15.64:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-20T12:55:46.363 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:46.361+0000 7f4bfce1f700 1 -- 172.21.15.71:0/2863775952 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f4be0031090 msgr2=0x7f4be0033550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:55:46.363 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:46.361+0000 7f4bfce1f700 1 --2- 172.21.15.71:0/2863775952 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f4be0031090 0x7f4be0033550 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 rx=0x7f4bec0096a0 tx=0x7f4bec006b40).stop 2024-02-20T12:55:46.363 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:46.361+0000 7f4bfce1f700 1 -- 172.21.15.71:0/2863775952 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4bf80f52d0 msgr2=0x7f4bf810b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:55:46.363 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:46.361+0000 7f4bfce1f700 1 --2- 172.21.15.71:0/2863775952 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4bf80f52d0 0x7f4bf810b730 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 rx=0x7f4be80089f0 tx=0x7f4be800b040).stop 2024-02-20T12:55:46.363 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:46.361+0000 7f4bfce1f700 1 -- 172.21.15.71:0/2863775952 shutdown_connections 2024-02-20T12:55:46.363 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:46.361+0000 7f4bfce1f700 1 --2- 172.21.15.71:0/2863775952 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f4be0031090 0x7f4be0033550 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:55:46.363 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:46.361+0000 7f4bfce1f700 1 --2- 172.21.15.71:0/2863775952 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4bf80f52d0 0x7f4bf810b730 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:55:46.363 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:46.361+0000 7f4bfce1f700 1 -- 172.21.15.71:0/2863775952 >> 172.21.15.71:0/2863775952 conn(0x7f4bf80f0a30 msgr2=0x7f4bf80f1550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:55:46.363 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:46.361+0000 7f4bfce1f700 1 -- 172.21.15.71:0/2863775952 shutdown_connections 2024-02-20T12:55:46.363 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:46.361+0000 7f4bfce1f700 1 -- 172.21.15.71:0/2863775952 wait complete. 2024-02-20T12:55:46.364 INFO:teuthology.orchestra.run.smithi071.stderr:dumped monmap epoch 1 2024-02-20T12:55:47.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:55:47 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:55:45.748959+0000 mgr.smithi064.vriyor (mgr.14156) 148 : cluster [DBG] pgmap v140: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:55:47.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:55:47 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:55:46.361433+0000 mon.smithi064 (mon.0) 183 : audit [DBG] from='client.? 172.21.15.71:0/2863775952' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-20T12:55:48.085 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-20T12:55:48.086 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/daemon-base:latest-pacific shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 680ee126-cfee-11ee-95bb-87774f69a715 -- ceph mon dump -f json 2024-02-20T12:55:49.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:55:49 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:55:47.749228+0000 mgr.smithi064.vriyor (mgr.14156) 149 : cluster [DBG] pgmap v141: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:55:50.046 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:50.044+0000 7f981e6c6700 1 -- 172.21.15.71:0/1339807880 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f98180602c0 msgr2=0x7f98180f9200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:55:50.123 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:50.044+0000 7f981e6c6700 1 --2- 172.21.15.71:0/1339807880 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f98180602c0 0x7f98180f9200 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 rx=0x7f9808009fb0 tx=0x7f9808009b30).stop 2024-02-20T12:55:50.123 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:50.044+0000 7f981e6c6700 1 -- 172.21.15.71:0/1339807880 shutdown_connections 2024-02-20T12:55:50.124 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:50.044+0000 7f981e6c6700 1 --2- 172.21.15.71:0/1339807880 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f98180602c0 0x7f98180f9200 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:55:50.124 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:50.044+0000 7f981e6c6700 1 -- 172.21.15.71:0/1339807880 >> 172.21.15.71:0/1339807880 conn(0x7f98180f0be0 msgr2=0x7f98180f3000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:55:50.124 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:50.044+0000 7f981e6c6700 1 -- 172.21.15.71:0/1339807880 shutdown_connections 2024-02-20T12:55:50.124 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:50.044+0000 7f981e6c6700 1 -- 172.21.15.71:0/1339807880 wait complete. 2024-02-20T12:55:50.124 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:50.045+0000 7f981e6c6700 1 Processor -- start 2024-02-20T12:55:50.128 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:50.045+0000 7f981e6c6700 1 -- start start 2024-02-20T12:55:50.128 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:50.045+0000 7f981e6c6700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f98180602c0 0x7f98180f6c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:55:50.129 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:50.045+0000 7f981e6c6700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98180f5100 con 0x7f98180602c0 2024-02-20T12:55:50.129 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:50.046+0000 7f9817fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f98180602c0 0x7f98180f6c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:55:50.129 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:50.046+0000 7f9817fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f98180602c0 0x7f98180f6c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.71:60066/0 (socket says 172.21.15.71:60066) 2024-02-20T12:55:50.129 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:50.046+0000 7f9817fff700 1 -- 172.21.15.71:0/3296786563 learned_addr learned my addr 172.21.15.71:0/3296786563 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-02-20T12:55:50.129 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:50.046+0000 7f9817fff700 1 -- 172.21.15.71:0/3296786563 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9808005040 con 0x7f98180602c0 2024-02-20T12:55:50.129 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:50.046+0000 7f9817fff700 1 --2- 172.21.15.71:0/3296786563 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f98180602c0 0x7f98180f6c80 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 rx=0x7f9808013fd0 tx=0x7f9808009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:55:50.129 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:50.046+0000 7f9814ff9700 1 -- 172.21.15.71:0/3296786563 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9808018070 con 0x7f98180602c0 2024-02-20T12:55:50.130 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:50.046+0000 7f981e6c6700 1 -- 172.21.15.71:0/3296786563 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98180f52a0 con 0x7f98180602c0 2024-02-20T12:55:50.130 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:50.046+0000 7f9814ff9700 1 -- 172.21.15.71:0/3296786563 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(25 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f9808007b10 con 0x7f98180602c0 2024-02-20T12:55:50.130 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:50.046+0000 7f9814ff9700 1 -- 172.21.15.71:0/3296786563 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f980801c3c0 con 0x7f98180602c0 2024-02-20T12:55:50.130 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:50.046+0000 7f981e6c6700 1 -- 172.21.15.71:0/3296786563 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98180f5730 con 0x7f98180602c0 2024-02-20T12:55:50.130 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:50.047+0000 7f9814ff9700 1 -- 172.21.15.71:0/3296786563 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 37151+0+0 (secure 0 0 0) 0x7f9808007c80 con 0x7f98180602c0 2024-02-20T12:55:50.130 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:50.047+0000 7f9814ff9700 1 --2- 172.21.15.71:0/3296786563 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f9800031090 0x7f9800033550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:55:50.130 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:50.047+0000 7f981e6c6700 1 -- 172.21.15.71:0/3296786563 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98180591e0 con 0x7f98180602c0 2024-02-20T12:55:50.131 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:50.048+0000 7f9814ff9700 1 -- 172.21.15.71:0/3296786563 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1065+0+0 (secure 0 0 0) 0x7f9808042970 con 0x7f98180602c0 2024-02-20T12:55:50.131 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:50.048+0000 7f98177fe700 1 --2- 172.21.15.71:0/3296786563 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f9800031090 0x7f9800033550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:55:50.131 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:50.048+0000 7f98177fe700 1 --2- 172.21.15.71:0/3296786563 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f9800031090 0x7f9800033550 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 rx=0x7f980c0096a0 tx=0x7f980c006b40).ready entity=mgr.14156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:55:50.131 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:50.052+0000 7f9814ff9700 1 -- 172.21.15.71:0/3296786563 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+153237 (secure 0 0 0) 0x7f9808025b40 con 0x7f98180602c0 2024-02-20T12:55:50.256 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:50.255+0000 7f981e6c6700 1 -- 172.21.15.71:0/3296786563 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f9818045bb0 con 0x7f98180602c0 2024-02-20T12:55:50.257 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:50.257+0000 7f9814ff9700 1 -- 172.21.15.71:0/3296786563 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7f9808025b40 con 0x7f98180602c0 2024-02-20T12:55:50.257 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-20T12:55:50.257 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":1,"fsid":"680ee126-cfee-11ee-95bb-87774f69a715","modified":"2024-02-20T12:49:44.699059Z","created":"2024-02-20T12:49:44.699059Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi064","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:3300","nonce":0},{"type":"v1","addr":"172.21.15.64:6789","nonce":0}]},"addr":"172.21.15.64:6789/0","public_addr":"172.21.15.64:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-20T12:55:50.259 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:50.257+0000 7f981e6c6700 1 -- 172.21.15.71:0/3296786563 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f9800031090 msgr2=0x7f9800033550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:55:50.259 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:50.257+0000 7f981e6c6700 1 --2- 172.21.15.71:0/3296786563 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f9800031090 0x7f9800033550 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 rx=0x7f980c0096a0 tx=0x7f980c006b40).stop 2024-02-20T12:55:50.259 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:50.257+0000 7f981e6c6700 1 -- 172.21.15.71:0/3296786563 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f98180602c0 msgr2=0x7f98180f6c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:55:50.259 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:50.257+0000 7f981e6c6700 1 --2- 172.21.15.71:0/3296786563 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f98180602c0 0x7f98180f6c80 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 rx=0x7f9808013fd0 tx=0x7f9808009b30).stop 2024-02-20T12:55:50.259 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:50.257+0000 7f981e6c6700 1 -- 172.21.15.71:0/3296786563 shutdown_connections 2024-02-20T12:55:50.259 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:50.257+0000 7f981e6c6700 1 --2- 172.21.15.71:0/3296786563 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f9800031090 0x7f9800033550 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:55:50.259 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:50.257+0000 7f981e6c6700 1 --2- 172.21.15.71:0/3296786563 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f98180602c0 0x7f98180f6c80 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:55:50.259 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:50.257+0000 7f981e6c6700 1 -- 172.21.15.71:0/3296786563 >> 172.21.15.71:0/3296786563 conn(0x7f98180f0be0 msgr2=0x7f98180f1630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:55:50.259 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:50.258+0000 7f981e6c6700 1 -- 172.21.15.71:0/3296786563 shutdown_connections 2024-02-20T12:55:50.260 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:50.258+0000 7f981e6c6700 1 -- 172.21.15.71:0/3296786563 wait complete. 2024-02-20T12:55:50.260 INFO:teuthology.orchestra.run.smithi071.stderr:dumped monmap epoch 1 2024-02-20T12:55:51.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:55:51 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:55:49.749542+0000 mgr.smithi064.vriyor (mgr.14156) 150 : cluster [DBG] pgmap v142: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:55:51.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:55:51 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:55:50.257521+0000 mon.smithi064 (mon.0) 184 : audit [DBG] from='client.? 172.21.15.71:0/3296786563' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-20T12:55:51.900 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-20T12:55:51.900 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/daemon-base:latest-pacific shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 680ee126-cfee-11ee-95bb-87774f69a715 -- ceph mon dump -f json 2024-02-20T12:55:53.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:55:53 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:55:51.749786+0000 mgr.smithi064.vriyor (mgr.14156) 151 : cluster [DBG] pgmap v143: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:55:53.854 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:53.826+0000 7fa1c140f700 1 -- 172.21.15.71:0/3623323732 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa1bc0f48f0 msgr2=0x7fa1bc0f4d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:55:53.854 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:53.826+0000 7fa1c140f700 1 --2- 172.21.15.71:0/3623323732 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa1bc0f48f0 0x7fa1bc0f4d10 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 rx=0x7fa1b0009fb0 tx=0x7fa1b0009b30).stop 2024-02-20T12:55:53.855 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:53.826+0000 7fa1c140f700 1 -- 172.21.15.71:0/3623323732 shutdown_connections 2024-02-20T12:55:53.855 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:53.826+0000 7fa1c140f700 1 --2- 172.21.15.71:0/3623323732 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa1bc0f48f0 0x7fa1bc0f4d10 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:55:53.855 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:53.826+0000 7fa1c140f700 1 -- 172.21.15.71:0/3623323732 >> 172.21.15.71:0/3623323732 conn(0x7fa1bc0f0390 msgr2=0x7fa1bc0f27b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:55:53.855 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:53.826+0000 7fa1c140f700 1 -- 172.21.15.71:0/3623323732 shutdown_connections 2024-02-20T12:55:53.855 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:53.826+0000 7fa1c140f700 1 -- 172.21.15.71:0/3623323732 wait complete. 2024-02-20T12:55:53.855 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:53.826+0000 7fa1c140f700 1 Processor -- start 2024-02-20T12:55:53.855 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:53.826+0000 7fa1c140f700 1 -- start start 2024-02-20T12:55:53.855 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:53.827+0000 7fa1c140f700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa1bc0f48f0 0x7fa1bc1069d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:55:53.855 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:53.827+0000 7fa1c140f700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1bc106ec0 con 0x7fa1bc0f48f0 2024-02-20T12:55:53.855 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:53.827+0000 7fa1baffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa1bc0f48f0 0x7fa1bc1069d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:55:53.855 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:53.827+0000 7fa1baffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa1bc0f48f0 0x7fa1bc1069d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.71:60080/0 (socket says 172.21.15.71:60080) 2024-02-20T12:55:53.855 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:53.827+0000 7fa1baffd700 1 -- 172.21.15.71:0/187156192 learned_addr learned my addr 172.21.15.71:0/187156192 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-02-20T12:55:53.855 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:53.827+0000 7fa1baffd700 1 -- 172.21.15.71:0/187156192 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1b0005040 con 0x7fa1bc0f48f0 2024-02-20T12:55:53.855 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:53.827+0000 7fa1baffd700 1 --2- 172.21.15.71:0/187156192 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa1bc0f48f0 0x7fa1bc1069d0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 rx=0x7fa1b0009f80 tx=0x7fa1b000b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:55:53.856 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:53.827+0000 7fa1a37fe700 1 -- 172.21.15.71:0/187156192 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa1b0009320 con 0x7fa1bc0f48f0 2024-02-20T12:55:53.856 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:53.827+0000 7fa1c140f700 1 -- 172.21.15.71:0/187156192 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1bc1070c0 con 0x7fa1bc0f48f0 2024-02-20T12:55:53.856 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:53.827+0000 7fa1a37fe700 1 -- 172.21.15.71:0/187156192 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(25 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7fa1b0013680 con 0x7fa1bc0f48f0 2024-02-20T12:55:53.856 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:53.827+0000 7fa1a37fe700 1 -- 172.21.15.71:0/187156192 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa1b001eda0 con 0x7fa1bc0f48f0 2024-02-20T12:55:53.856 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:53.827+0000 7fa1c140f700 1 -- 172.21.15.71:0/187156192 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1bc107550 con 0x7fa1bc0f48f0 2024-02-20T12:55:53.856 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:53.828+0000 7fa1a37fe700 1 -- 172.21.15.71:0/187156192 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 37151+0+0 (secure 0 0 0) 0x7fa1b001e420 con 0x7fa1bc0f48f0 2024-02-20T12:55:53.856 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:53.828+0000 7fa1c140f700 1 -- 172.21.15.71:0/187156192 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1bc045bb0 con 0x7fa1bc0f48f0 2024-02-20T12:55:53.856 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:53.828+0000 7fa1a37fe700 1 --2- 172.21.15.71:0/187156192 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fa1a4031090 0x7fa1a4033550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:55:53.856 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:53.828+0000 7fa1a37fe700 1 -- 172.21.15.71:0/187156192 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1065+0+0 (secure 0 0 0) 0x7fa1b0042450 con 0x7fa1bc0f48f0 2024-02-20T12:55:53.856 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:53.829+0000 7fa1ba7fc700 1 --2- 172.21.15.71:0/187156192 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fa1a4031090 0x7fa1a4033550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:55:53.856 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:53.829+0000 7fa1ba7fc700 1 --2- 172.21.15.71:0/187156192 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fa1a4031090 0x7fa1a4033550 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 rx=0x7fa1ac0096a0 tx=0x7fa1ac006b40).ready entity=mgr.14156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:55:53.856 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:53.833+0000 7fa1a37fe700 1 -- 172.21.15.71:0/187156192 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+153237 (secure 0 0 0) 0x7fa1b001d380 con 0x7fa1bc0f48f0 2024-02-20T12:55:54.041 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:54.040+0000 7fa1c140f700 1 -- 172.21.15.71:0/187156192 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fa1bc05f450 con 0x7fa1bc0f48f0 2024-02-20T12:55:54.042 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:54.041+0000 7fa1a37fe700 1 -- 172.21.15.71:0/187156192 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7fa1b001cb10 con 0x7fa1bc0f48f0 2024-02-20T12:55:54.043 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-20T12:55:54.043 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":1,"fsid":"680ee126-cfee-11ee-95bb-87774f69a715","modified":"2024-02-20T12:49:44.699059Z","created":"2024-02-20T12:49:44.699059Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi064","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:3300","nonce":0},{"type":"v1","addr":"172.21.15.64:6789","nonce":0}]},"addr":"172.21.15.64:6789/0","public_addr":"172.21.15.64:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-20T12:55:54.045 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:54.042+0000 7fa1c140f700 1 -- 172.21.15.71:0/187156192 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fa1a4031090 msgr2=0x7fa1a4033550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:55:54.045 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:54.042+0000 7fa1c140f700 1 --2- 172.21.15.71:0/187156192 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fa1a4031090 0x7fa1a4033550 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 rx=0x7fa1ac0096a0 tx=0x7fa1ac006b40).stop 2024-02-20T12:55:54.045 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:54.042+0000 7fa1c140f700 1 -- 172.21.15.71:0/187156192 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa1bc0f48f0 msgr2=0x7fa1bc1069d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:55:54.045 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:54.042+0000 7fa1c140f700 1 --2- 172.21.15.71:0/187156192 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa1bc0f48f0 0x7fa1bc1069d0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 rx=0x7fa1b0009f80 tx=0x7fa1b000b040).stop 2024-02-20T12:55:54.045 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:54.042+0000 7fa1c140f700 1 -- 172.21.15.71:0/187156192 shutdown_connections 2024-02-20T12:55:54.046 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:54.042+0000 7fa1c140f700 1 --2- 172.21.15.71:0/187156192 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fa1a4031090 0x7fa1a4033550 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:55:54.046 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:54.042+0000 7fa1c140f700 1 --2- 172.21.15.71:0/187156192 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa1bc0f48f0 0x7fa1bc1069d0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:55:54.046 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:54.042+0000 7fa1c140f700 1 -- 172.21.15.71:0/187156192 >> 172.21.15.71:0/187156192 conn(0x7fa1bc0f0390 msgr2=0x7fa1bc0f0e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:55:54.046 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:54.042+0000 7fa1c140f700 1 -- 172.21.15.71:0/187156192 shutdown_connections 2024-02-20T12:55:54.046 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:54.042+0000 7fa1c140f700 1 -- 172.21.15.71:0/187156192 wait complete. 2024-02-20T12:55:54.046 INFO:teuthology.orchestra.run.smithi071.stderr:dumped monmap epoch 1 2024-02-20T12:55:54.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:55:54 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:55:53.847235+0000 mon.smithi064 (mon.0) 185 : audit [INF] from='mgr.14156 172.21.15.64:0/3284974785' entity='mgr.smithi064.vriyor' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi064.vriyor/mirror_snapshot_schedule"}]: dispatch 2024-02-20T12:55:54.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:55:54 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:55:53.848032+0000 mon.smithi064 (mon.0) 186 : audit [INF] from='mgr.14156 172.21.15.64:0/3284974785' entity='mgr.smithi064.vriyor' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi064.vriyor/trash_purge_schedule"}]: dispatch 2024-02-20T12:55:54.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:55:54 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:55:54.042340+0000 mon.smithi064 (mon.0) 187 : audit [DBG] from='client.? 172.21.15.71:0/187156192' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-20T12:55:55.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:55:55 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:55:53.750084+0000 mgr.smithi064.vriyor (mgr.14156) 152 : cluster [DBG] pgmap v144: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:55:55.723 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-20T12:55:55.723 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/daemon-base:latest-pacific shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 680ee126-cfee-11ee-95bb-87774f69a715 -- ceph mon dump -f json 2024-02-20T12:55:57.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:55:57 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:55:55.750306+0000 mgr.smithi064.vriyor (mgr.14156) 153 : cluster [DBG] pgmap v145: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:55:57.710 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:57.706+0000 7f349ba0e700 1 -- 172.21.15.71:0/1760113622 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f34940f6590 msgr2=0x7f34940f69b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:55:57.710 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:57.706+0000 7f349ba0e700 1 --2- 172.21.15.71:0/1760113622 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f34940f6590 0x7f34940f69b0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 rx=0x7f3484009fb0 tx=0x7f3484009b30).stop 2024-02-20T12:55:57.710 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:57.707+0000 7f349ba0e700 1 -- 172.21.15.71:0/1760113622 shutdown_connections 2024-02-20T12:55:57.710 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:57.707+0000 7f349ba0e700 1 --2- 172.21.15.71:0/1760113622 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f34940f6590 0x7f34940f69b0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:55:57.710 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:57.707+0000 7f349ba0e700 1 -- 172.21.15.71:0/1760113622 >> 172.21.15.71:0/1760113622 conn(0x7f34940f1d10 msgr2=0x7f34940f4130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:55:57.710 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:57.707+0000 7f349ba0e700 1 -- 172.21.15.71:0/1760113622 shutdown_connections 2024-02-20T12:55:57.710 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:57.707+0000 7f349ba0e700 1 -- 172.21.15.71:0/1760113622 wait complete. 2024-02-20T12:55:57.710 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:57.707+0000 7f349ba0e700 1 Processor -- start 2024-02-20T12:55:57.710 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:57.707+0000 7f349ba0e700 1 -- start start 2024-02-20T12:55:57.710 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:57.707+0000 7f349ba0e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f34940f6590 0x7f34941067d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:55:57.710 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:57.707+0000 7f349ba0e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3494106cc0 con 0x7f34940f6590 2024-02-20T12:55:57.710 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:57.708+0000 7f34997aa700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f34940f6590 0x7f34941067d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:55:57.711 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:57.708+0000 7f34997aa700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f34940f6590 0x7f34941067d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.71:40512/0 (socket says 172.21.15.71:40512) 2024-02-20T12:55:57.711 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:57.708+0000 7f34997aa700 1 -- 172.21.15.71:0/4117896522 learned_addr learned my addr 172.21.15.71:0/4117896522 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-02-20T12:55:57.711 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:57.708+0000 7f34997aa700 1 -- 172.21.15.71:0/4117896522 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3484005040 con 0x7f34940f6590 2024-02-20T12:55:57.711 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:57.708+0000 7f34997aa700 1 --2- 172.21.15.71:0/4117896522 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f34940f6590 0x7f34941067d0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 rx=0x7f34840089f0 tx=0x7f348400b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:55:57.711 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:57.708+0000 7f3489ffb700 1 -- 172.21.15.71:0/4117896522 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3484009320 con 0x7f34940f6590 2024-02-20T12:55:57.711 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:57.708+0000 7f349ba0e700 1 -- 172.21.15.71:0/4117896522 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3494106ec0 con 0x7f34940f6590 2024-02-20T12:55:57.711 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:57.708+0000 7f3489ffb700 1 -- 172.21.15.71:0/4117896522 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(25 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f3484013ec0 con 0x7f34940f6590 2024-02-20T12:55:57.711 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:57.708+0000 7f3489ffb700 1 -- 172.21.15.71:0/4117896522 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3484014d30 con 0x7f34940f6590 2024-02-20T12:55:57.711 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:57.708+0000 7f349ba0e700 1 -- 172.21.15.71:0/4117896522 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3494107350 con 0x7f34940f6590 2024-02-20T12:55:57.712 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:57.709+0000 7f3489ffb700 1 -- 172.21.15.71:0/4117896522 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 37151+0+0 (secure 0 0 0) 0x7f34840095f0 con 0x7f34940f6590 2024-02-20T12:55:57.712 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:57.709+0000 7f3489ffb700 1 --2- 172.21.15.71:0/4117896522 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f3480031090 0x7f3480033550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:55:57.712 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:57.709+0000 7f349ba0e700 1 -- 172.21.15.71:0/4117896522 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3494045bb0 con 0x7f34940f6590 2024-02-20T12:55:57.712 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:57.709+0000 7f3489ffb700 1 -- 172.21.15.71:0/4117896522 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1065+0+0 (secure 0 0 0) 0x7f3484042420 con 0x7f34940f6590 2024-02-20T12:55:57.712 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:57.710+0000 7f3498fa9700 1 --2- 172.21.15.71:0/4117896522 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f3480031090 0x7f3480033550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:55:57.712 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:57.710+0000 7f3498fa9700 1 --2- 172.21.15.71:0/4117896522 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f3480031090 0x7f3480033550 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 rx=0x7f34900096a0 tx=0x7f3490006b40).ready entity=mgr.14156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:55:57.715 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:57.715+0000 7f3489ffb700 1 -- 172.21.15.71:0/4117896522 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+153237 (secure 0 0 0) 0x7f348400fe40 con 0x7f34940f6590 2024-02-20T12:55:57.921 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:57.919+0000 7f349ba0e700 1 -- 172.21.15.71:0/4117896522 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f34940fc170 con 0x7f34940f6590 2024-02-20T12:55:57.921 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:57.920+0000 7f3489ffb700 1 -- 172.21.15.71:0/4117896522 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7f348400fe40 con 0x7f34940f6590 2024-02-20T12:55:57.922 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-20T12:55:57.922 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":1,"fsid":"680ee126-cfee-11ee-95bb-87774f69a715","modified":"2024-02-20T12:49:44.699059Z","created":"2024-02-20T12:49:44.699059Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi064","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:3300","nonce":0},{"type":"v1","addr":"172.21.15.64:6789","nonce":0}]},"addr":"172.21.15.64:6789/0","public_addr":"172.21.15.64:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-20T12:55:57.923 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:57.922+0000 7f349ba0e700 1 -- 172.21.15.71:0/4117896522 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f3480031090 msgr2=0x7f3480033550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:55:57.923 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:57.922+0000 7f349ba0e700 1 --2- 172.21.15.71:0/4117896522 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f3480031090 0x7f3480033550 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 rx=0x7f34900096a0 tx=0x7f3490006b40).stop 2024-02-20T12:55:57.923 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:57.922+0000 7f349ba0e700 1 -- 172.21.15.71:0/4117896522 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f34940f6590 msgr2=0x7f34941067d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:55:57.923 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:57.922+0000 7f349ba0e700 1 --2- 172.21.15.71:0/4117896522 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f34940f6590 0x7f34941067d0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 rx=0x7f34840089f0 tx=0x7f348400b040).stop 2024-02-20T12:55:57.923 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:57.922+0000 7f349ba0e700 1 -- 172.21.15.71:0/4117896522 shutdown_connections 2024-02-20T12:55:57.924 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:57.922+0000 7f349ba0e700 1 --2- 172.21.15.71:0/4117896522 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f3480031090 0x7f3480033550 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:55:57.924 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:57.922+0000 7f349ba0e700 1 --2- 172.21.15.71:0/4117896522 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f34940f6590 0x7f34941067d0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:55:57.924 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:57.922+0000 7f349ba0e700 1 -- 172.21.15.71:0/4117896522 >> 172.21.15.71:0/4117896522 conn(0x7f34940f1d10 msgr2=0x7f34940f27d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:55:57.924 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:57.922+0000 7f349ba0e700 1 -- 172.21.15.71:0/4117896522 shutdown_connections 2024-02-20T12:55:57.924 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:55:57.922+0000 7f349ba0e700 1 -- 172.21.15.71:0/4117896522 wait complete. 2024-02-20T12:55:57.924 INFO:teuthology.orchestra.run.smithi071.stderr:dumped monmap epoch 1 2024-02-20T12:55:58.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:55:58 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:55:57.921797+0000 mon.smithi064 (mon.0) 188 : audit [DBG] from='client.? 172.21.15.71:0/4117896522' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-20T12:55:59.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:55:59 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:55:57.750667+0000 mgr.smithi064.vriyor (mgr.14156) 154 : cluster [DBG] pgmap v146: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:55:59.824 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-20T12:55:59.825 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/daemon-base:latest-pacific shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 680ee126-cfee-11ee-95bb-87774f69a715 -- ceph mon dump -f json 2024-02-20T12:56:01.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:56:01 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:55:59.751071+0000 mgr.smithi064.vriyor (mgr.14156) 155 : cluster [DBG] pgmap v147: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:56:01.734 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:01.732+0000 7f54790be700 1 -- 172.21.15.71:0/3225489435 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f54740f5480 msgr2=0x7f54740f58a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:56:01.734 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:01.732+0000 7f54790be700 1 --2- 172.21.15.71:0/3225489435 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f54740f5480 0x7f54740f58a0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 rx=0x7f5468009fb0 tx=0x7f5468009b30).stop 2024-02-20T12:56:01.734 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:01.733+0000 7f54790be700 1 -- 172.21.15.71:0/3225489435 shutdown_connections 2024-02-20T12:56:01.734 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:01.733+0000 7f54790be700 1 --2- 172.21.15.71:0/3225489435 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f54740f5480 0x7f54740f58a0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:56:01.734 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:01.733+0000 7f54790be700 1 -- 172.21.15.71:0/3225489435 >> 172.21.15.71:0/3225489435 conn(0x7f54740f0be0 msgr2=0x7f54740f3000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:56:01.735 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:01.733+0000 7f54790be700 1 -- 172.21.15.71:0/3225489435 shutdown_connections 2024-02-20T12:56:01.735 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:01.733+0000 7f54790be700 1 -- 172.21.15.71:0/3225489435 wait complete. 2024-02-20T12:56:01.735 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:01.733+0000 7f54790be700 1 Processor -- start 2024-02-20T12:56:01.735 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:01.733+0000 7f54790be700 1 -- start start 2024-02-20T12:56:01.735 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:01.734+0000 7f54790be700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f54740602e0 0x7f5474109600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:56:01.735 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:01.734+0000 7f54790be700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5474109af0 con 0x7f54740602e0 2024-02-20T12:56:01.737 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:01.734+0000 7f5472d9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f54740602e0 0x7f5474109600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:56:01.737 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:01.734+0000 7f5472d9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f54740602e0 0x7f5474109600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.71:40526/0 (socket says 172.21.15.71:40526) 2024-02-20T12:56:01.737 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:01.734+0000 7f5472d9d700 1 -- 172.21.15.71:0/3131443785 learned_addr learned my addr 172.21.15.71:0/3131443785 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-02-20T12:56:01.737 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:01.734+0000 7f5472d9d700 1 -- 172.21.15.71:0/3131443785 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5468005040 con 0x7f54740602e0 2024-02-20T12:56:01.737 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:01.734+0000 7f5472d9d700 1 --2- 172.21.15.71:0/3131443785 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f54740602e0 0x7f5474109600 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 rx=0x7f5468009880 tx=0x7f5468009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:56:01.737 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:01.734+0000 7f545b7fe700 1 -- 172.21.15.71:0/3131443785 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5468013680 con 0x7f54740602e0 2024-02-20T12:56:01.737 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:01.734+0000 7f54790be700 1 -- 172.21.15.71:0/3131443785 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5474109c30 con 0x7f54740602e0 2024-02-20T12:56:01.737 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:01.735+0000 7f545b7fe700 1 -- 172.21.15.71:0/3131443785 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(25 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f54680095f0 con 0x7f54740602e0 2024-02-20T12:56:01.738 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:01.735+0000 7f545b7fe700 1 -- 172.21.15.71:0/3131443785 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5468014e40 con 0x7f54740602e0 2024-02-20T12:56:01.738 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:01.735+0000 7f54790be700 1 -- 172.21.15.71:0/3131443785 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f547410a060 con 0x7f54740602e0 2024-02-20T12:56:01.738 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:01.735+0000 7f545b7fe700 1 -- 172.21.15.71:0/3131443785 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 37151+0+0 (secure 0 0 0) 0x7f5468014420 con 0x7f54740602e0 2024-02-20T12:56:01.738 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:01.735+0000 7f54790be700 1 -- 172.21.15.71:0/3131443785 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54740591e0 con 0x7f54740602e0 2024-02-20T12:56:01.738 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:01.735+0000 7f545b7fe700 1 --2- 172.21.15.71:0/3131443785 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f545c031090 0x7f545c033550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:56:01.738 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:01.735+0000 7f545b7fe700 1 -- 172.21.15.71:0/3131443785 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1065+0+0 (secure 0 0 0) 0x7f54680424d0 con 0x7f54740602e0 2024-02-20T12:56:01.738 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:01.736+0000 7f547259c700 1 --2- 172.21.15.71:0/3131443785 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f545c031090 0x7f545c033550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:56:01.738 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:01.736+0000 7f547259c700 1 --2- 172.21.15.71:0/3131443785 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f545c031090 0x7f545c033550 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 rx=0x7f54640096a0 tx=0x7f5464006b40).ready entity=mgr.14156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:56:01.740 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:01.740+0000 7f545b7fe700 1 -- 172.21.15.71:0/3131443785 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+153237 (secure 0 0 0) 0x7f5468025cc0 con 0x7f54740602e0 2024-02-20T12:56:01.946 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:01.945+0000 7f54790be700 1 -- 172.21.15.71:0/3131443785 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f5474045bb0 con 0x7f54740602e0 2024-02-20T12:56:01.946 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:01.946+0000 7f545b7fe700 1 -- 172.21.15.71:0/3131443785 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7f5468025ae0 con 0x7f54740602e0 2024-02-20T12:56:01.946 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-20T12:56:01.947 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":1,"fsid":"680ee126-cfee-11ee-95bb-87774f69a715","modified":"2024-02-20T12:49:44.699059Z","created":"2024-02-20T12:49:44.699059Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi064","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:3300","nonce":0},{"type":"v1","addr":"172.21.15.64:6789","nonce":0}]},"addr":"172.21.15.64:6789/0","public_addr":"172.21.15.64:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-20T12:56:01.948 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:01.948+0000 7f54790be700 1 -- 172.21.15.71:0/3131443785 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f545c031090 msgr2=0x7f545c033550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:56:01.948 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:01.948+0000 7f54790be700 1 --2- 172.21.15.71:0/3131443785 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f545c031090 0x7f545c033550 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 rx=0x7f54640096a0 tx=0x7f5464006b40).stop 2024-02-20T12:56:01.948 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:01.948+0000 7f54790be700 1 -- 172.21.15.71:0/3131443785 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f54740602e0 msgr2=0x7f5474109600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:56:01.949 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:01.948+0000 7f54790be700 1 --2- 172.21.15.71:0/3131443785 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f54740602e0 0x7f5474109600 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 rx=0x7f5468009880 tx=0x7f5468009b30).stop 2024-02-20T12:56:01.949 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:01.948+0000 7f54790be700 1 -- 172.21.15.71:0/3131443785 shutdown_connections 2024-02-20T12:56:01.949 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:01.948+0000 7f54790be700 1 --2- 172.21.15.71:0/3131443785 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f545c031090 0x7f545c033550 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:56:01.949 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:01.948+0000 7f54790be700 1 --2- 172.21.15.71:0/3131443785 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f54740602e0 0x7f5474109600 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:56:01.949 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:01.948+0000 7f54790be700 1 -- 172.21.15.71:0/3131443785 >> 172.21.15.71:0/3131443785 conn(0x7f54740f0be0 msgr2=0x7f54740f1700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:56:01.949 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:01.948+0000 7f54790be700 1 -- 172.21.15.71:0/3131443785 shutdown_connections 2024-02-20T12:56:01.949 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:01.948+0000 7f54790be700 1 -- 172.21.15.71:0/3131443785 wait complete. 2024-02-20T12:56:01.949 INFO:teuthology.orchestra.run.smithi071.stderr:dumped monmap epoch 1 2024-02-20T12:56:02.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:56:02 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:56:01.946720+0000 mon.smithi064 (mon.0) 189 : audit [DBG] from='client.? 172.21.15.71:0/3131443785' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-20T12:56:03.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:56:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:56:01.751340+0000 mgr.smithi064.vriyor (mgr.14156) 156 : cluster [DBG] pgmap v148: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:56:03.585 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-20T12:56:03.585 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/daemon-base:latest-pacific shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 680ee126-cfee-11ee-95bb-87774f69a715 -- ceph mon dump -f json 2024-02-20T12:56:05.491 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:05.489+0000 7f9b1f510700 1 -- 172.21.15.71:0/283102393 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9b180f5480 msgr2=0x7f9b180f58a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:56:05.491 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:05.489+0000 7f9b1f510700 1 --2- 172.21.15.71:0/283102393 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9b180f5480 0x7f9b180f58a0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 rx=0x7f9b14009fb0 tx=0x7f9b14009b30).stop 2024-02-20T12:56:05.491 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:05.489+0000 7f9b1f510700 1 -- 172.21.15.71:0/283102393 shutdown_connections 2024-02-20T12:56:05.491 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:05.489+0000 7f9b1f510700 1 --2- 172.21.15.71:0/283102393 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9b180f5480 0x7f9b180f58a0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:56:05.491 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:05.489+0000 7f9b1f510700 1 -- 172.21.15.71:0/283102393 >> 172.21.15.71:0/283102393 conn(0x7f9b180f0be0 msgr2=0x7f9b180f3000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:56:05.491 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:05.489+0000 7f9b1f510700 1 -- 172.21.15.71:0/283102393 shutdown_connections 2024-02-20T12:56:05.491 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:05.489+0000 7f9b1f510700 1 -- 172.21.15.71:0/283102393 wait complete. 2024-02-20T12:56:05.491 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:05.490+0000 7f9b1f510700 1 Processor -- start 2024-02-20T12:56:05.491 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:05.490+0000 7f9b1f510700 1 -- start start 2024-02-20T12:56:05.491 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:05.490+0000 7f9b1f510700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9b180f5480 0x7f9b18109510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:56:05.491 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:05.490+0000 7f9b1f510700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b18060300 con 0x7f9b180f5480 2024-02-20T12:56:05.492 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:05.490+0000 7f9b1d2ac700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9b180f5480 0x7f9b18109510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:56:05.492 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:05.490+0000 7f9b1d2ac700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9b180f5480 0x7f9b18109510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.71:40550/0 (socket says 172.21.15.71:40550) 2024-02-20T12:56:05.492 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:05.490+0000 7f9b1d2ac700 1 -- 172.21.15.71:0/2836604065 learned_addr learned my addr 172.21.15.71:0/2836604065 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-02-20T12:56:05.493 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:05.491+0000 7f9b1d2ac700 1 -- 172.21.15.71:0/2836604065 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b14005040 con 0x7f9b180f5480 2024-02-20T12:56:05.493 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:05.491+0000 7f9b1d2ac700 1 --2- 172.21.15.71:0/2836604065 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9b180f5480 0x7f9b18109510 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 rx=0x7f9b140089f0 tx=0x7f9b1400b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:56:05.493 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:05.491+0000 7f9b0dffb700 1 -- 172.21.15.71:0/2836604065 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9b14009320 con 0x7f9b180f5480 2024-02-20T12:56:05.493 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:05.491+0000 7f9b1f510700 1 -- 172.21.15.71:0/2836604065 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b18060500 con 0x7f9b180f5480 2024-02-20T12:56:05.494 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:05.491+0000 7f9b0dffb700 1 -- 172.21.15.71:0/2836604065 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(25 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f9b14013ec0 con 0x7f9b180f5480 2024-02-20T12:56:05.494 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:05.491+0000 7f9b0dffb700 1 -- 172.21.15.71:0/2836604065 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9b14014d30 con 0x7f9b180f5480 2024-02-20T12:56:05.494 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:05.491+0000 7f9b1f510700 1 -- 172.21.15.71:0/2836604065 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b18109bc0 con 0x7f9b180f5480 2024-02-20T12:56:05.494 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:05.492+0000 7f9b0dffb700 1 -- 172.21.15.71:0/2836604065 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 37151+0+0 (secure 0 0 0) 0x7f9b140095f0 con 0x7f9b180f5480 2024-02-20T12:56:05.494 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:05.492+0000 7f9b1f510700 1 -- 172.21.15.71:0/2836604065 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b180591e0 con 0x7f9b180f5480 2024-02-20T12:56:05.494 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:05.492+0000 7f9b0dffb700 1 --2- 172.21.15.71:0/2836604065 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f9b040310e0 0x7f9b040335a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:56:05.494 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:05.492+0000 7f9b0dffb700 1 -- 172.21.15.71:0/2836604065 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1065+0+0 (secure 0 0 0) 0x7f9b140424e0 con 0x7f9b180f5480 2024-02-20T12:56:05.494 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:05.493+0000 7f9b1caab700 1 --2- 172.21.15.71:0/2836604065 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f9b040310e0 0x7f9b040335a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:56:05.494 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:05.493+0000 7f9b1caab700 1 --2- 172.21.15.71:0/2836604065 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f9b040310e0 0x7f9b040335a0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 rx=0x7f9b080096a0 tx=0x7f9b08006b40).ready entity=mgr.14156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:56:05.497 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:05.497+0000 7f9b0dffb700 1 -- 172.21.15.71:0/2836604065 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+153237 (secure 0 0 0) 0x7f9b1401d7f0 con 0x7f9b180f5480 2024-02-20T12:56:05.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:56:05 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:56:03.751634+0000 mgr.smithi064.vriyor (mgr.14156) 157 : cluster [DBG] pgmap v149: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:56:05.703 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:05.703+0000 7f9b1f510700 1 -- 172.21.15.71:0/2836604065 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f9b18045bb0 con 0x7f9b180f5480 2024-02-20T12:56:05.704 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:05.703+0000 7f9b0dffb700 1 -- 172.21.15.71:0/2836604065 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7f9b1400fe40 con 0x7f9b180f5480 2024-02-20T12:56:05.704 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-20T12:56:05.708 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":1,"fsid":"680ee126-cfee-11ee-95bb-87774f69a715","modified":"2024-02-20T12:49:44.699059Z","created":"2024-02-20T12:49:44.699059Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi064","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:3300","nonce":0},{"type":"v1","addr":"172.21.15.64:6789","nonce":0}]},"addr":"172.21.15.64:6789/0","public_addr":"172.21.15.64:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-20T12:56:05.709 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:05.705+0000 7f9b1f510700 1 -- 172.21.15.71:0/2836604065 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f9b040310e0 msgr2=0x7f9b040335a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:56:05.709 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:05.705+0000 7f9b1f510700 1 --2- 172.21.15.71:0/2836604065 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f9b040310e0 0x7f9b040335a0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 rx=0x7f9b080096a0 tx=0x7f9b08006b40).stop 2024-02-20T12:56:05.709 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:05.705+0000 7f9b1f510700 1 -- 172.21.15.71:0/2836604065 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9b180f5480 msgr2=0x7f9b18109510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:56:05.709 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:05.705+0000 7f9b1f510700 1 --2- 172.21.15.71:0/2836604065 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9b180f5480 0x7f9b18109510 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 rx=0x7f9b140089f0 tx=0x7f9b1400b040).stop 2024-02-20T12:56:05.709 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:05.705+0000 7f9b1f510700 1 -- 172.21.15.71:0/2836604065 shutdown_connections 2024-02-20T12:56:05.709 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:05.705+0000 7f9b1f510700 1 --2- 172.21.15.71:0/2836604065 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f9b040310e0 0x7f9b040335a0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:56:05.709 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:05.705+0000 7f9b1f510700 1 --2- 172.21.15.71:0/2836604065 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9b180f5480 0x7f9b18109510 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:56:05.709 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:05.705+0000 7f9b1f510700 1 -- 172.21.15.71:0/2836604065 >> 172.21.15.71:0/2836604065 conn(0x7f9b180f0be0 msgr2=0x7f9b180f16d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:56:05.709 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:05.705+0000 7f9b1f510700 1 -- 172.21.15.71:0/2836604065 shutdown_connections 2024-02-20T12:56:05.709 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:05.705+0000 7f9b1f510700 1 -- 172.21.15.71:0/2836604065 wait complete. 2024-02-20T12:56:05.709 INFO:teuthology.orchestra.run.smithi071.stderr:dumped monmap epoch 1 2024-02-20T12:56:06.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:56:06 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:56:05.704310+0000 mon.smithi064 (mon.0) 190 : audit [DBG] from='client.? 172.21.15.71:0/2836604065' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-20T12:56:07.340 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-20T12:56:07.340 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/daemon-base:latest-pacific shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 680ee126-cfee-11ee-95bb-87774f69a715 -- ceph mon dump -f json 2024-02-20T12:56:07.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:56:07 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:56:05.751957+0000 mgr.smithi064.vriyor (mgr.14156) 158 : cluster [DBG] pgmap v150: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:56:09.236 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:09.199+0000 7f577e920700 1 -- 172.21.15.71:0/912716450 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f57780f54b0 msgr2=0x7f57780f58d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:56:09.236 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:09.199+0000 7f577e920700 1 --2- 172.21.15.71:0/912716450 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f57780f54b0 0x7f57780f58d0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 rx=0x7f5760009fb0 tx=0x7f5760009b30).stop 2024-02-20T12:56:09.236 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:09.199+0000 7f577e920700 1 -- 172.21.15.71:0/912716450 shutdown_connections 2024-02-20T12:56:09.236 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:09.199+0000 7f577e920700 1 --2- 172.21.15.71:0/912716450 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f57780f54b0 0x7f57780f58d0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:56:09.237 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:09.199+0000 7f577e920700 1 -- 172.21.15.71:0/912716450 >> 172.21.15.71:0/912716450 conn(0x7f57780f0c10 msgr2=0x7f57780f3030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:56:09.237 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:09.199+0000 7f577e920700 1 -- 172.21.15.71:0/912716450 shutdown_connections 2024-02-20T12:56:09.237 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:09.199+0000 7f577e920700 1 -- 172.21.15.71:0/912716450 wait complete. 2024-02-20T12:56:09.237 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:09.199+0000 7f577e920700 1 Processor -- start 2024-02-20T12:56:09.237 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:09.200+0000 7f577e920700 1 -- start start 2024-02-20T12:56:09.237 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:09.200+0000 7f577e920700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f57780f54b0 0x7f57781094e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:56:09.237 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:09.200+0000 7f577e920700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5778060360 con 0x7f57780f54b0 2024-02-20T12:56:09.237 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:09.200+0000 7f5777fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f57780f54b0 0x7f57781094e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:56:09.237 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:09.200+0000 7f5777fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f57780f54b0 0x7f57781094e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.71:48278/0 (socket says 172.21.15.71:48278) 2024-02-20T12:56:09.237 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:09.200+0000 7f5777fff700 1 -- 172.21.15.71:0/3262782744 learned_addr learned my addr 172.21.15.71:0/3262782744 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-02-20T12:56:09.237 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:09.200+0000 7f5777fff700 1 -- 172.21.15.71:0/3262782744 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5760005040 con 0x7f57780f54b0 2024-02-20T12:56:09.237 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:09.200+0000 7f5777fff700 1 --2- 172.21.15.71:0/3262782744 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f57780f54b0 0x7f57781094e0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 rx=0x7f57600089f0 tx=0x7f576000b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:56:09.237 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:09.201+0000 7f5774ff9700 1 -- 172.21.15.71:0/3262782744 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5760009320 con 0x7f57780f54b0 2024-02-20T12:56:09.237 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:09.201+0000 7f577e920700 1 -- 172.21.15.71:0/3262782744 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5778060560 con 0x7f57780f54b0 2024-02-20T12:56:09.238 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:09.201+0000 7f5774ff9700 1 -- 172.21.15.71:0/3262782744 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(25 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f5760013ec0 con 0x7f57780f54b0 2024-02-20T12:56:09.238 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:09.201+0000 7f577e920700 1 -- 172.21.15.71:0/3262782744 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5778109c10 con 0x7f57780f54b0 2024-02-20T12:56:09.238 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:09.201+0000 7f5774ff9700 1 -- 172.21.15.71:0/3262782744 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5760014d50 con 0x7f57780f54b0 2024-02-20T12:56:09.238 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:09.202+0000 7f5774ff9700 1 -- 172.21.15.71:0/3262782744 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 37151+0+0 (secure 0 0 0) 0x7f576001e420 con 0x7f57780f54b0 2024-02-20T12:56:09.238 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:09.202+0000 7f5774ff9700 1 --2- 172.21.15.71:0/3262782744 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f5764031090 0x7f5764033550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:56:09.238 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:09.202+0000 7f5774ff9700 1 -- 172.21.15.71:0/3262782744 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1065+0+0 (secure 0 0 0) 0x7f5760042370 con 0x7f57780f54b0 2024-02-20T12:56:09.238 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:09.202+0000 7f577e920700 1 -- 172.21.15.71:0/3262782744 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57780591e0 con 0x7f57780f54b0 2024-02-20T12:56:09.238 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:09.202+0000 7f57777fe700 1 --2- 172.21.15.71:0/3262782744 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f5764031090 0x7f5764033550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:56:09.238 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:09.202+0000 7f57777fe700 1 --2- 172.21.15.71:0/3262782744 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f5764031090 0x7f5764033550 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 rx=0x7f57680096a0 tx=0x7f5768006b40).ready entity=mgr.14156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:56:09.238 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:09.207+0000 7f5774ff9700 1 -- 172.21.15.71:0/3262782744 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+153237 (secure 0 0 0) 0x7f57600273f0 con 0x7f57780f54b0 2024-02-20T12:56:09.417 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:09.416+0000 7f577e920700 1 -- 172.21.15.71:0/3262782744 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f57780fafa0 con 0x7f57780f54b0 2024-02-20T12:56:09.417 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:09.416+0000 7f5774ff9700 1 -- 172.21.15.71:0/3262782744 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7f576001c320 con 0x7f57780f54b0 2024-02-20T12:56:09.417 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-20T12:56:09.417 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":1,"fsid":"680ee126-cfee-11ee-95bb-87774f69a715","modified":"2024-02-20T12:49:44.699059Z","created":"2024-02-20T12:49:44.699059Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi064","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:3300","nonce":0},{"type":"v1","addr":"172.21.15.64:6789","nonce":0}]},"addr":"172.21.15.64:6789/0","public_addr":"172.21.15.64:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-20T12:56:09.418 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:09.418+0000 7f577e920700 1 -- 172.21.15.71:0/3262782744 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f5764031090 msgr2=0x7f5764033550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:56:09.418 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:09.418+0000 7f577e920700 1 --2- 172.21.15.71:0/3262782744 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f5764031090 0x7f5764033550 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 rx=0x7f57680096a0 tx=0x7f5768006b40).stop 2024-02-20T12:56:09.418 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:09.418+0000 7f577e920700 1 -- 172.21.15.71:0/3262782744 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f57780f54b0 msgr2=0x7f57781094e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:56:09.418 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:09.418+0000 7f577e920700 1 --2- 172.21.15.71:0/3262782744 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f57780f54b0 0x7f57781094e0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 rx=0x7f57600089f0 tx=0x7f576000b040).stop 2024-02-20T12:56:09.419 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:09.418+0000 7f577e920700 1 -- 172.21.15.71:0/3262782744 shutdown_connections 2024-02-20T12:56:09.419 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:09.418+0000 7f577e920700 1 --2- 172.21.15.71:0/3262782744 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f5764031090 0x7f5764033550 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:56:09.419 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:09.418+0000 7f577e920700 1 --2- 172.21.15.71:0/3262782744 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f57780f54b0 0x7f57781094e0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:56:09.419 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:09.418+0000 7f577e920700 1 -- 172.21.15.71:0/3262782744 >> 172.21.15.71:0/3262782744 conn(0x7f57780f0c10 msgr2=0x7f57780f1700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:56:09.419 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:09.418+0000 7f577e920700 1 -- 172.21.15.71:0/3262782744 shutdown_connections 2024-02-20T12:56:09.419 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:09.418+0000 7f577e920700 1 -- 172.21.15.71:0/3262782744 wait complete. 2024-02-20T12:56:09.419 INFO:teuthology.orchestra.run.smithi071.stderr:dumped monmap epoch 1 2024-02-20T12:56:09.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:56:09 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:56:07.752183+0000 mgr.smithi064.vriyor (mgr.14156) 159 : cluster [DBG] pgmap v151: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:56:10.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:56:10 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:56:09.417026+0000 mon.smithi064 (mon.0) 191 : audit [DBG] from='client.? 172.21.15.71:0/3262782744' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-20T12:56:11.121 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-20T12:56:11.121 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/daemon-base:latest-pacific shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 680ee126-cfee-11ee-95bb-87774f69a715 -- ceph mon dump -f json 2024-02-20T12:56:11.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:56:11 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:56:09.752483+0000 mgr.smithi064.vriyor (mgr.14156) 160 : cluster [DBG] pgmap v152: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:56:12.999 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:12.997+0000 7fe661ca0700 1 -- 172.21.15.71:0/304701608 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe65c060300 msgr2=0x7fe65c0f91c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:56:12.999 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:12.997+0000 7fe661ca0700 1 --2- 172.21.15.71:0/304701608 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe65c060300 0x7fe65c0f91c0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 rx=0x7fe64c009fb0 tx=0x7fe64c009b30).stop 2024-02-20T12:56:12.999 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:12.997+0000 7fe661ca0700 1 -- 172.21.15.71:0/304701608 shutdown_connections 2024-02-20T12:56:12.999 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:12.997+0000 7fe661ca0700 1 --2- 172.21.15.71:0/304701608 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe65c060300 0x7fe65c0f91c0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:56:12.999 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:12.997+0000 7fe661ca0700 1 -- 172.21.15.71:0/304701608 >> 172.21.15.71:0/304701608 conn(0x7fe65c0f0be0 msgr2=0x7fe65c0f3000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:56:12.999 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:12.997+0000 7fe661ca0700 1 -- 172.21.15.71:0/304701608 shutdown_connections 2024-02-20T12:56:12.999 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:12.997+0000 7fe661ca0700 1 -- 172.21.15.71:0/304701608 wait complete. 2024-02-20T12:56:13.000 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:12.998+0000 7fe661ca0700 1 Processor -- start 2024-02-20T12:56:13.000 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:12.998+0000 7fe661ca0700 1 -- start start 2024-02-20T12:56:13.000 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:12.998+0000 7fe661ca0700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe65c060300 0x7fe65c0f6c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:56:13.000 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:12.998+0000 7fe661ca0700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe65c0f50c0 con 0x7fe65c060300 2024-02-20T12:56:13.000 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:12.998+0000 7fe65b7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe65c060300 0x7fe65c0f6c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:56:13.000 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:12.998+0000 7fe65b7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe65c060300 0x7fe65c0f6c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.71:48288/0 (socket says 172.21.15.71:48288) 2024-02-20T12:56:13.000 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:12.998+0000 7fe65b7fe700 1 -- 172.21.15.71:0/231305910 learned_addr learned my addr 172.21.15.71:0/231305910 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-02-20T12:56:13.000 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:12.998+0000 7fe65b7fe700 1 -- 172.21.15.71:0/231305910 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe64c005040 con 0x7fe65c060300 2024-02-20T12:56:13.000 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:12.999+0000 7fe65b7fe700 1 --2- 172.21.15.71:0/231305910 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe65c060300 0x7fe65c0f6c40 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 rx=0x7fe64c009f80 tx=0x7fe64c00b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:56:13.000 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:12.999+0000 7fe653fff700 1 -- 172.21.15.71:0/231305910 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe64c009320 con 0x7fe65c060300 2024-02-20T12:56:13.000 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:12.999+0000 7fe661ca0700 1 -- 172.21.15.71:0/231305910 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe65c0f5260 con 0x7fe65c060300 2024-02-20T12:56:13.000 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:12.999+0000 7fe653fff700 1 -- 172.21.15.71:0/231305910 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(25 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7fe64c013680 con 0x7fe65c060300 2024-02-20T12:56:13.000 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:12.999+0000 7fe653fff700 1 -- 172.21.15.71:0/231305910 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe64c01eda0 con 0x7fe65c060300 2024-02-20T12:56:13.001 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:12.999+0000 7fe661ca0700 1 -- 172.21.15.71:0/231305910 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe65c0f56f0 con 0x7fe65c060300 2024-02-20T12:56:13.001 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:13.000+0000 7fe653fff700 1 -- 172.21.15.71:0/231305910 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 37151+0+0 (secure 0 0 0) 0x7fe64c01e420 con 0x7fe65c060300 2024-02-20T12:56:13.002 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:13.000+0000 7fe661ca0700 1 -- 172.21.15.71:0/231305910 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe65c0591e0 con 0x7fe65c060300 2024-02-20T12:56:13.002 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:13.000+0000 7fe653fff700 1 --2- 172.21.15.71:0/231305910 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fe648031090 0x7fe648033550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:56:13.002 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:13.000+0000 7fe653fff700 1 -- 172.21.15.71:0/231305910 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1065+0+0 (secure 0 0 0) 0x7fe64c042370 con 0x7fe65c060300 2024-02-20T12:56:13.002 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:13.000+0000 7fe65affd700 1 --2- 172.21.15.71:0/231305910 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fe648031090 0x7fe648033550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:56:13.002 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:13.001+0000 7fe65affd700 1 --2- 172.21.15.71:0/231305910 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fe648031090 0x7fe648033550 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 rx=0x7fe6440096a0 tx=0x7fe644006b40).ready entity=mgr.14156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:56:13.005 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:13.005+0000 7fe653fff700 1 -- 172.21.15.71:0/231305910 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+153237 (secure 0 0 0) 0x7fe64c01ccf0 con 0x7fe65c060300 2024-02-20T12:56:13.209 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:13.209+0000 7fe661ca0700 1 -- 172.21.15.71:0/231305910 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fe65c0fc060 con 0x7fe65c060300 2024-02-20T12:56:13.210 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:13.209+0000 7fe653fff700 1 -- 172.21.15.71:0/231305910 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7fe64c01cb10 con 0x7fe65c060300 2024-02-20T12:56:13.210 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-20T12:56:13.210 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":1,"fsid":"680ee126-cfee-11ee-95bb-87774f69a715","modified":"2024-02-20T12:49:44.699059Z","created":"2024-02-20T12:49:44.699059Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi064","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:3300","nonce":0},{"type":"v1","addr":"172.21.15.64:6789","nonce":0}]},"addr":"172.21.15.64:6789/0","public_addr":"172.21.15.64:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-20T12:56:13.211 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:13.211+0000 7fe661ca0700 1 -- 172.21.15.71:0/231305910 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fe648031090 msgr2=0x7fe648033550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:56:13.211 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:13.211+0000 7fe661ca0700 1 --2- 172.21.15.71:0/231305910 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fe648031090 0x7fe648033550 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 rx=0x7fe6440096a0 tx=0x7fe644006b40).stop 2024-02-20T12:56:13.212 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:13.211+0000 7fe661ca0700 1 -- 172.21.15.71:0/231305910 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe65c060300 msgr2=0x7fe65c0f6c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:56:13.212 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:13.211+0000 7fe661ca0700 1 --2- 172.21.15.71:0/231305910 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe65c060300 0x7fe65c0f6c40 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 rx=0x7fe64c009f80 tx=0x7fe64c00b040).stop 2024-02-20T12:56:13.212 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:13.211+0000 7fe661ca0700 1 -- 172.21.15.71:0/231305910 shutdown_connections 2024-02-20T12:56:13.212 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:13.211+0000 7fe661ca0700 1 --2- 172.21.15.71:0/231305910 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fe648031090 0x7fe648033550 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:56:13.212 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:13.211+0000 7fe661ca0700 1 --2- 172.21.15.71:0/231305910 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe65c060300 0x7fe65c0f6c40 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:56:13.212 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:13.211+0000 7fe661ca0700 1 -- 172.21.15.71:0/231305910 >> 172.21.15.71:0/231305910 conn(0x7fe65c0f0be0 msgr2=0x7fe65c0f1720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:56:13.212 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:13.211+0000 7fe661ca0700 1 -- 172.21.15.71:0/231305910 shutdown_connections 2024-02-20T12:56:13.212 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:13.211+0000 7fe661ca0700 1 -- 172.21.15.71:0/231305910 wait complete. 2024-02-20T12:56:13.212 INFO:teuthology.orchestra.run.smithi071.stderr:dumped monmap epoch 1 2024-02-20T12:56:13.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:56:13 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:56:11.752744+0000 mgr.smithi064.vriyor (mgr.14156) 161 : cluster [DBG] pgmap v153: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:56:13.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:56:13 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:56:13.210197+0000 mon.smithi064 (mon.0) 192 : audit [DBG] from='client.? 172.21.15.71:0/231305910' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-20T12:56:14.844 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-20T12:56:14.845 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/daemon-base:latest-pacific shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 680ee126-cfee-11ee-95bb-87774f69a715 -- ceph mon dump -f json 2024-02-20T12:56:15.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:56:15 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:56:13.753022+0000 mgr.smithi064.vriyor (mgr.14156) 162 : cluster [DBG] pgmap v154: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:56:16.661 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:16.658+0000 7fbceaa8a700 1 -- 172.21.15.71:0/3900700351 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbce40f54b0 msgr2=0x7fbce40f58d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:56:16.661 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:16.658+0000 7fbceaa8a700 1 --2- 172.21.15.71:0/3900700351 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbce40f54b0 0x7fbce40f58d0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 rx=0x7fbcd8009fb0 tx=0x7fbcd8009b30).stop 2024-02-20T12:56:16.661 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:16.658+0000 7fbceaa8a700 1 -- 172.21.15.71:0/3900700351 shutdown_connections 2024-02-20T12:56:16.661 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:16.658+0000 7fbceaa8a700 1 --2- 172.21.15.71:0/3900700351 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbce40f54b0 0x7fbce40f58d0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:56:16.661 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:16.658+0000 7fbceaa8a700 1 -- 172.21.15.71:0/3900700351 >> 172.21.15.71:0/3900700351 conn(0x7fbce40f0c10 msgr2=0x7fbce40f3030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:56:16.661 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:16.658+0000 7fbceaa8a700 1 -- 172.21.15.71:0/3900700351 shutdown_connections 2024-02-20T12:56:16.661 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:16.658+0000 7fbceaa8a700 1 -- 172.21.15.71:0/3900700351 wait complete. 2024-02-20T12:56:16.661 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:16.659+0000 7fbceaa8a700 1 Processor -- start 2024-02-20T12:56:16.661 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:16.659+0000 7fbceaa8a700 1 -- start start 2024-02-20T12:56:16.662 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:16.659+0000 7fbceaa8a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbce40f54b0 0x7fbce40fd490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:56:16.662 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:16.659+0000 7fbceaa8a700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbce40fb910 con 0x7fbce40f54b0 2024-02-20T12:56:16.662 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:16.659+0000 7fbce8826700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbce40f54b0 0x7fbce40fd490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:56:16.662 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:16.660+0000 7fbce8826700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbce40f54b0 0x7fbce40fd490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.71:48292/0 (socket says 172.21.15.71:48292) 2024-02-20T12:56:16.662 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:16.660+0000 7fbce8826700 1 -- 172.21.15.71:0/460413942 learned_addr learned my addr 172.21.15.71:0/460413942 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-02-20T12:56:16.662 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:16.660+0000 7fbce8826700 1 -- 172.21.15.71:0/460413942 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcd8005040 con 0x7fbce40f54b0 2024-02-20T12:56:16.662 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:16.660+0000 7fbce8826700 1 --2- 172.21.15.71:0/460413942 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbce40f54b0 0x7fbce40fd490 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 rx=0x7fbcd80089f0 tx=0x7fbcd800b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:56:16.662 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:16.660+0000 7fbce17fa700 1 -- 172.21.15.71:0/460413942 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fbcd8009320 con 0x7fbce40f54b0 2024-02-20T12:56:16.662 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:16.660+0000 7fbceaa8a700 1 -- 172.21.15.71:0/460413942 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbce40fbab0 con 0x7fbce40f54b0 2024-02-20T12:56:16.662 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:16.660+0000 7fbce17fa700 1 -- 172.21.15.71:0/460413942 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(25 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7fbcd8013ec0 con 0x7fbce40f54b0 2024-02-20T12:56:16.662 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:16.660+0000 7fbce17fa700 1 -- 172.21.15.71:0/460413942 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fbcd8014d30 con 0x7fbce40f54b0 2024-02-20T12:56:16.662 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:16.660+0000 7fbceaa8a700 1 -- 172.21.15.71:0/460413942 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbce40fbf40 con 0x7fbce40f54b0 2024-02-20T12:56:16.663 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:16.661+0000 7fbce17fa700 1 -- 172.21.15.71:0/460413942 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 37151+0+0 (secure 0 0 0) 0x7fbcd80095f0 con 0x7fbce40f54b0 2024-02-20T12:56:16.664 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:16.661+0000 7fbceaa8a700 1 -- 172.21.15.71:0/460413942 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbce40591e0 con 0x7fbce40f54b0 2024-02-20T12:56:16.664 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:16.661+0000 7fbce17fa700 1 --2- 172.21.15.71:0/460413942 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fbcd4031090 0x7fbcd4033550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:56:16.664 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:16.661+0000 7fbce17fa700 1 -- 172.21.15.71:0/460413942 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1065+0+0 (secure 0 0 0) 0x7fbcd80424a0 con 0x7fbce40f54b0 2024-02-20T12:56:16.664 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:16.661+0000 7fbce3fff700 1 --2- 172.21.15.71:0/460413942 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fbcd4031090 0x7fbcd4033550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:56:16.664 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:16.662+0000 7fbce3fff700 1 --2- 172.21.15.71:0/460413942 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fbcd4031090 0x7fbcd4033550 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 rx=0x7fbcd00096a0 tx=0x7fbcd0006b40).ready entity=mgr.14156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:56:16.666 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:16.666+0000 7fbce17fa700 1 -- 172.21.15.71:0/460413942 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+153237 (secure 0 0 0) 0x7fbcd801d7f0 con 0x7fbce40f54b0 2024-02-20T12:56:16.872 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:16.872+0000 7fbceaa8a700 1 -- 172.21.15.71:0/460413942 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fbce40fafa0 con 0x7fbce40f54b0 2024-02-20T12:56:16.873 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:16.873+0000 7fbce17fa700 1 -- 172.21.15.71:0/460413942 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7fbcd801d7f0 con 0x7fbce40f54b0 2024-02-20T12:56:16.873 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-20T12:56:16.873 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":1,"fsid":"680ee126-cfee-11ee-95bb-87774f69a715","modified":"2024-02-20T12:49:44.699059Z","created":"2024-02-20T12:49:44.699059Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi064","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:3300","nonce":0},{"type":"v1","addr":"172.21.15.64:6789","nonce":0}]},"addr":"172.21.15.64:6789/0","public_addr":"172.21.15.64:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-20T12:56:16.875 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:16.874+0000 7fbceaa8a700 1 -- 172.21.15.71:0/460413942 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fbcd4031090 msgr2=0x7fbcd4033550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:56:16.875 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:16.874+0000 7fbceaa8a700 1 --2- 172.21.15.71:0/460413942 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fbcd4031090 0x7fbcd4033550 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 rx=0x7fbcd00096a0 tx=0x7fbcd0006b40).stop 2024-02-20T12:56:16.875 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:16.874+0000 7fbceaa8a700 1 -- 172.21.15.71:0/460413942 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbce40f54b0 msgr2=0x7fbce40fd490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:56:16.875 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:16.874+0000 7fbceaa8a700 1 --2- 172.21.15.71:0/460413942 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbce40f54b0 0x7fbce40fd490 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 rx=0x7fbcd80089f0 tx=0x7fbcd800b040).stop 2024-02-20T12:56:16.875 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:16.874+0000 7fbceaa8a700 1 -- 172.21.15.71:0/460413942 shutdown_connections 2024-02-20T12:56:16.875 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:16.874+0000 7fbceaa8a700 1 --2- 172.21.15.71:0/460413942 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fbcd4031090 0x7fbcd4033550 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:56:16.875 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:16.874+0000 7fbceaa8a700 1 --2- 172.21.15.71:0/460413942 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbce40f54b0 0x7fbce40fd490 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:56:16.875 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:16.874+0000 7fbceaa8a700 1 -- 172.21.15.71:0/460413942 >> 172.21.15.71:0/460413942 conn(0x7fbce40f0c10 msgr2=0x7fbce405fee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:56:16.875 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:16.874+0000 7fbceaa8a700 1 -- 172.21.15.71:0/460413942 shutdown_connections 2024-02-20T12:56:16.876 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:16.874+0000 7fbceaa8a700 1 -- 172.21.15.71:0/460413942 wait complete. 2024-02-20T12:56:16.876 INFO:teuthology.orchestra.run.smithi071.stderr:dumped monmap epoch 1 2024-02-20T12:56:17.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:56:17 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:56:15.753341+0000 mgr.smithi064.vriyor (mgr.14156) 163 : cluster [DBG] pgmap v155: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:56:17.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:56:17 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:56:16.873478+0000 mon.smithi064 (mon.0) 193 : audit [DBG] from='client.? 172.21.15.71:0/460413942' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-20T12:56:18.423 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-20T12:56:18.423 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/daemon-base:latest-pacific shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 680ee126-cfee-11ee-95bb-87774f69a715 -- ceph mon dump -f json 2024-02-20T12:56:19.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:56:19 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:56:17.753719+0000 mgr.smithi064.vriyor (mgr.14156) 164 : cluster [DBG] pgmap v156: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:56:20.426 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:20.421+0000 7f683d907700 1 -- 172.21.15.71:0/2297403713 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f68380f6590 msgr2=0x7f68380f69b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:56:20.426 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:20.421+0000 7f683d907700 1 --2- 172.21.15.71:0/2297403713 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f68380f6590 0x7f68380f69b0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 rx=0x7f682c009fb0 tx=0x7f682c009b30).stop 2024-02-20T12:56:20.426 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:20.421+0000 7f683d907700 1 -- 172.21.15.71:0/2297403713 shutdown_connections 2024-02-20T12:56:20.426 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:20.421+0000 7f683d907700 1 --2- 172.21.15.71:0/2297403713 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f68380f6590 0x7f68380f69b0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:56:20.427 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:20.421+0000 7f683d907700 1 -- 172.21.15.71:0/2297403713 >> 172.21.15.71:0/2297403713 conn(0x7f68380f1cf0 msgr2=0x7f68380f4110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:56:20.427 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:20.421+0000 7f683d907700 1 -- 172.21.15.71:0/2297403713 shutdown_connections 2024-02-20T12:56:20.427 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:20.421+0000 7f683d907700 1 -- 172.21.15.71:0/2297403713 wait complete. 2024-02-20T12:56:20.427 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:20.422+0000 7f683d907700 1 Processor -- start 2024-02-20T12:56:20.427 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:20.422+0000 7f683d907700 1 -- start start 2024-02-20T12:56:20.427 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:20.422+0000 7f683d907700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f68380f6590 0x7f68381067d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:56:20.428 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:20.422+0000 7f683d907700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6838106cc0 con 0x7f68380f6590 2024-02-20T12:56:20.433 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:20.422+0000 7f6836ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f68380f6590 0x7f68381067d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:56:20.433 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:20.422+0000 7f6836ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f68380f6590 0x7f68381067d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.71:38626/0 (socket says 172.21.15.71:38626) 2024-02-20T12:56:20.433 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:20.422+0000 7f6836ffd700 1 -- 172.21.15.71:0/3663965270 learned_addr learned my addr 172.21.15.71:0/3663965270 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-02-20T12:56:20.433 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:20.422+0000 7f6836ffd700 1 -- 172.21.15.71:0/3663965270 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f682c005040 con 0x7f68380f6590 2024-02-20T12:56:20.433 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:20.422+0000 7f6836ffd700 1 --2- 172.21.15.71:0/3663965270 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f68380f6590 0x7f68381067d0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 rx=0x7f682c0089f0 tx=0x7f682c00b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:56:20.433 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:20.423+0000 7f681f7fe700 1 -- 172.21.15.71:0/3663965270 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f682c009320 con 0x7f68380f6590 2024-02-20T12:56:20.434 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:20.423+0000 7f683d907700 1 -- 172.21.15.71:0/3663965270 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6838106ec0 con 0x7f68380f6590 2024-02-20T12:56:20.434 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:20.423+0000 7f681f7fe700 1 -- 172.21.15.71:0/3663965270 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(25 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f682c013ec0 con 0x7f68380f6590 2024-02-20T12:56:20.434 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:20.423+0000 7f681f7fe700 1 -- 172.21.15.71:0/3663965270 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f682c014d30 con 0x7f68380f6590 2024-02-20T12:56:20.434 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:20.423+0000 7f683d907700 1 -- 172.21.15.71:0/3663965270 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6838107350 con 0x7f68380f6590 2024-02-20T12:56:20.434 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:20.423+0000 7f681f7fe700 1 -- 172.21.15.71:0/3663965270 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 37151+0+0 (secure 0 0 0) 0x7f682c0095f0 con 0x7f68380f6590 2024-02-20T12:56:20.434 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:20.424+0000 7f683d907700 1 -- 172.21.15.71:0/3663965270 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6838045bb0 con 0x7f68380f6590 2024-02-20T12:56:20.434 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:20.424+0000 7f681f7fe700 1 --2- 172.21.15.71:0/3663965270 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f6820031090 0x7f6820033550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:56:20.434 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:20.424+0000 7f681f7fe700 1 -- 172.21.15.71:0/3663965270 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1065+0+0 (secure 0 0 0) 0x7f682c0424a0 con 0x7f68380f6590 2024-02-20T12:56:20.434 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:20.424+0000 7f68367fc700 1 --2- 172.21.15.71:0/3663965270 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f6820031090 0x7f6820033550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:56:20.434 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:20.424+0000 7f68367fc700 1 --2- 172.21.15.71:0/3663965270 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f6820031090 0x7f6820033550 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 rx=0x7f68280096a0 tx=0x7f6828006b40).ready entity=mgr.14156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:56:20.435 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:20.428+0000 7f681f7fe700 1 -- 172.21.15.71:0/3663965270 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+153237 (secure 0 0 0) 0x7f682c01d7f0 con 0x7f68380f6590 2024-02-20T12:56:20.638 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:20.638+0000 7f683d907700 1 -- 172.21.15.71:0/3663965270 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f6838108200 con 0x7f68380f6590 2024-02-20T12:56:20.639 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:20.639+0000 7f681f7fe700 1 -- 172.21.15.71:0/3663965270 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7f682c01d7f0 con 0x7f68380f6590 2024-02-20T12:56:20.639 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-20T12:56:20.639 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":1,"fsid":"680ee126-cfee-11ee-95bb-87774f69a715","modified":"2024-02-20T12:49:44.699059Z","created":"2024-02-20T12:49:44.699059Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi064","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:3300","nonce":0},{"type":"v1","addr":"172.21.15.64:6789","nonce":0}]},"addr":"172.21.15.64:6789/0","public_addr":"172.21.15.64:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-20T12:56:20.641 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:20.640+0000 7f683d907700 1 -- 172.21.15.71:0/3663965270 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f6820031090 msgr2=0x7f6820033550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:56:20.641 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:20.640+0000 7f683d907700 1 --2- 172.21.15.71:0/3663965270 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f6820031090 0x7f6820033550 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 rx=0x7f68280096a0 tx=0x7f6828006b40).stop 2024-02-20T12:56:20.641 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:20.640+0000 7f683d907700 1 -- 172.21.15.71:0/3663965270 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f68380f6590 msgr2=0x7f68381067d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:56:20.641 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:20.640+0000 7f683d907700 1 --2- 172.21.15.71:0/3663965270 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f68380f6590 0x7f68381067d0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 rx=0x7f682c0089f0 tx=0x7f682c00b040).stop 2024-02-20T12:56:20.641 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:20.640+0000 7f683d907700 1 -- 172.21.15.71:0/3663965270 shutdown_connections 2024-02-20T12:56:20.641 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:20.640+0000 7f683d907700 1 --2- 172.21.15.71:0/3663965270 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f6820031090 0x7f6820033550 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:56:20.641 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:20.640+0000 7f683d907700 1 --2- 172.21.15.71:0/3663965270 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f68380f6590 0x7f68381067d0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:56:20.641 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:20.640+0000 7f683d907700 1 -- 172.21.15.71:0/3663965270 >> 172.21.15.71:0/3663965270 conn(0x7f68380f1cf0 msgr2=0x7f68380f2770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:56:20.641 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:20.640+0000 7f683d907700 1 -- 172.21.15.71:0/3663965270 shutdown_connections 2024-02-20T12:56:20.641 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:20.640+0000 7f683d907700 1 -- 172.21.15.71:0/3663965270 wait complete. 2024-02-20T12:56:20.642 INFO:teuthology.orchestra.run.smithi071.stderr:dumped monmap epoch 1 2024-02-20T12:56:21.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:56:21 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:56:19.754085+0000 mgr.smithi064.vriyor (mgr.14156) 165 : cluster [DBG] pgmap v157: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:56:21.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:56:21 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:56:20.639434+0000 mon.smithi064 (mon.0) 194 : audit [DBG] from='client.? 172.21.15.71:0/3663965270' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-20T12:56:22.263 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-20T12:56:22.263 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/daemon-base:latest-pacific shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 680ee126-cfee-11ee-95bb-87774f69a715 -- ceph mon dump -f json 2024-02-20T12:56:23.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:56:23 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:56:21.754421+0000 mgr.smithi064.vriyor (mgr.14156) 166 : cluster [DBG] pgmap v158: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:56:24.188 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:24.168+0000 7f6d246d3700 1 -- 172.21.15.71:0/1589472543 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6d1c0f6590 msgr2=0x7f6d1c0f69b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:56:24.189 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:24.168+0000 7f6d246d3700 1 --2- 172.21.15.71:0/1589472543 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6d1c0f6590 0x7f6d1c0f69b0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 rx=0x7f6d0c004760 tx=0x7f6d0c009ad0).stop 2024-02-20T12:56:24.189 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:24.169+0000 7f6d246d3700 1 -- 172.21.15.71:0/1589472543 shutdown_connections 2024-02-20T12:56:24.189 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:24.169+0000 7f6d246d3700 1 --2- 172.21.15.71:0/1589472543 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6d1c0f6590 0x7f6d1c0f69b0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:56:24.189 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:24.169+0000 7f6d246d3700 1 -- 172.21.15.71:0/1589472543 >> 172.21.15.71:0/1589472543 conn(0x7f6d1c0f1cd0 msgr2=0x7f6d1c0f4110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:56:24.189 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:24.169+0000 7f6d246d3700 1 -- 172.21.15.71:0/1589472543 shutdown_connections 2024-02-20T12:56:24.189 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:24.169+0000 7f6d246d3700 1 -- 172.21.15.71:0/1589472543 wait complete. 2024-02-20T12:56:24.189 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:24.169+0000 7f6d246d3700 1 Processor -- start 2024-02-20T12:56:24.189 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:24.169+0000 7f6d246d3700 1 -- start start 2024-02-20T12:56:24.189 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:24.169+0000 7f6d246d3700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6d1c0f6590 0x7f6d1c1067c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:56:24.189 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:24.169+0000 7f6d246d3700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d1c106cb0 con 0x7f6d1c0f6590 2024-02-20T12:56:24.189 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:24.169+0000 7f6d2246f700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6d1c0f6590 0x7f6d1c1067c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:56:24.189 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:24.170+0000 7f6d2246f700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6d1c0f6590 0x7f6d1c1067c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.71:38634/0 (socket says 172.21.15.71:38634) 2024-02-20T12:56:24.189 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:24.170+0000 7f6d2246f700 1 -- 172.21.15.71:0/993132090 learned_addr learned my addr 172.21.15.71:0/993132090 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-02-20T12:56:24.189 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:24.170+0000 7f6d2246f700 1 -- 172.21.15.71:0/993132090 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d0c009d70 con 0x7f6d1c0f6590 2024-02-20T12:56:24.190 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:24.170+0000 7f6d2246f700 1 --2- 172.21.15.71:0/993132090 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6d1c0f6590 0x7f6d1c1067c0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 rx=0x7f6d0c013f10 tx=0x7f6d0c009ad0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:56:24.190 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:24.170+0000 7f6d12ffd700 1 -- 172.21.15.71:0/993132090 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6d0c00e040 con 0x7f6d1c0f6590 2024-02-20T12:56:24.190 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:24.170+0000 7f6d246d3700 1 -- 172.21.15.71:0/993132090 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d1c106eb0 con 0x7f6d1c0f6590 2024-02-20T12:56:24.190 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:24.170+0000 7f6d12ffd700 1 -- 172.21.15.71:0/993132090 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(25 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f6d0c007df0 con 0x7f6d1c0f6590 2024-02-20T12:56:24.190 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:24.170+0000 7f6d12ffd700 1 -- 172.21.15.71:0/993132090 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6d0c014d60 con 0x7f6d1c0f6590 2024-02-20T12:56:24.190 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:24.170+0000 7f6d246d3700 1 -- 172.21.15.71:0/993132090 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d1c107340 con 0x7f6d1c0f6590 2024-02-20T12:56:24.190 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:24.171+0000 7f6d12ffd700 1 -- 172.21.15.71:0/993132090 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 37151+0+0 (secure 0 0 0) 0x7f6d0c014520 con 0x7f6d1c0f6590 2024-02-20T12:56:24.190 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:24.171+0000 7f6d246d3700 1 -- 172.21.15.71:0/993132090 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d1c045bb0 con 0x7f6d1c0f6590 2024-02-20T12:56:24.190 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:24.171+0000 7f6d12ffd700 1 --2- 172.21.15.71:0/993132090 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f6d080310e0 0x7f6d080335a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:56:24.191 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:24.171+0000 7f6d12ffd700 1 -- 172.21.15.71:0/993132090 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1065+0+0 (secure 0 0 0) 0x7f6d0c042580 con 0x7f6d1c0f6590 2024-02-20T12:56:24.191 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:24.171+0000 7f6d21c6e700 1 --2- 172.21.15.71:0/993132090 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f6d080310e0 0x7f6d080335a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:56:24.191 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:24.172+0000 7f6d21c6e700 1 --2- 172.21.15.71:0/993132090 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f6d080310e0 0x7f6d080335a0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 rx=0x7f6d180096a0 tx=0x7f6d18006b40).ready entity=mgr.14156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:56:24.191 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:24.176+0000 7f6d12ffd700 1 -- 172.21.15.71:0/993132090 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+153237 (secure 0 0 0) 0x7f6d0c007960 con 0x7f6d1c0f6590 2024-02-20T12:56:24.386 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:24.385+0000 7f6d246d3700 1 -- 172.21.15.71:0/993132090 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f6d1c108190 con 0x7f6d1c0f6590 2024-02-20T12:56:24.530 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:24.386+0000 7f6d12ffd700 1 -- 172.21.15.71:0/993132090 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7f6d0c0262f0 con 0x7f6d1c0f6590 2024-02-20T12:56:24.530 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-20T12:56:24.531 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":1,"fsid":"680ee126-cfee-11ee-95bb-87774f69a715","modified":"2024-02-20T12:49:44.699059Z","created":"2024-02-20T12:49:44.699059Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi064","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:3300","nonce":0},{"type":"v1","addr":"172.21.15.64:6789","nonce":0}]},"addr":"172.21.15.64:6789/0","public_addr":"172.21.15.64:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-20T12:56:24.531 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:24.387+0000 7f6d246d3700 1 -- 172.21.15.71:0/993132090 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f6d080310e0 msgr2=0x7f6d080335a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:56:24.531 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:24.387+0000 7f6d246d3700 1 --2- 172.21.15.71:0/993132090 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f6d080310e0 0x7f6d080335a0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 rx=0x7f6d180096a0 tx=0x7f6d18006b40).stop 2024-02-20T12:56:24.531 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:24.387+0000 7f6d246d3700 1 -- 172.21.15.71:0/993132090 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6d1c0f6590 msgr2=0x7f6d1c1067c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:56:24.532 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:24.387+0000 7f6d246d3700 1 --2- 172.21.15.71:0/993132090 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6d1c0f6590 0x7f6d1c1067c0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 rx=0x7f6d0c013f10 tx=0x7f6d0c009ad0).stop 2024-02-20T12:56:24.532 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:24.387+0000 7f6d246d3700 1 -- 172.21.15.71:0/993132090 shutdown_connections 2024-02-20T12:56:24.532 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:24.387+0000 7f6d246d3700 1 --2- 172.21.15.71:0/993132090 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f6d080310e0 0x7f6d080335a0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:56:24.532 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:24.387+0000 7f6d246d3700 1 --2- 172.21.15.71:0/993132090 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6d1c0f6590 0x7f6d1c1067c0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:56:24.532 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:24.387+0000 7f6d246d3700 1 -- 172.21.15.71:0/993132090 >> 172.21.15.71:0/993132090 conn(0x7f6d1c0f1cd0 msgr2=0x7f6d1c0f27c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:56:24.532 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:24.387+0000 7f6d246d3700 1 -- 172.21.15.71:0/993132090 shutdown_connections 2024-02-20T12:56:24.532 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:24.387+0000 7f6d246d3700 1 -- 172.21.15.71:0/993132090 wait complete. 2024-02-20T12:56:24.532 INFO:teuthology.orchestra.run.smithi071.stderr:dumped monmap epoch 1 2024-02-20T12:56:25.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:56:25 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:56:23.754792+0000 mgr.smithi064.vriyor (mgr.14156) 167 : cluster [DBG] pgmap v159: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:56:25.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:56:25 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:56:24.386653+0000 mon.smithi064 (mon.0) 195 : audit [DBG] from='client.? 172.21.15.71:0/993132090' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-20T12:56:26.039 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-20T12:56:26.039 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/daemon-base:latest-pacific shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 680ee126-cfee-11ee-95bb-87774f69a715 -- ceph mon dump -f json 2024-02-20T12:56:27.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:56:27 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:56:25.755135+0000 mgr.smithi064.vriyor (mgr.14156) 168 : cluster [DBG] pgmap v160: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:56:27.883 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:27.826+0000 7f3f88939700 1 -- 172.21.15.71:0/1941368255 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3f840f63e0 msgr2=0x7f3f840f6800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:56:27.883 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:27.826+0000 7f3f88939700 1 --2- 172.21.15.71:0/1941368255 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3f840f63e0 0x7f3f840f6800 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 rx=0x7f3f78009fb0 tx=0x7f3f78009b30).stop 2024-02-20T12:56:27.883 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:27.826+0000 7f3f88939700 1 -- 172.21.15.71:0/1941368255 shutdown_connections 2024-02-20T12:56:27.883 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:27.826+0000 7f3f88939700 1 --2- 172.21.15.71:0/1941368255 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3f840f63e0 0x7f3f840f6800 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:56:27.883 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:27.826+0000 7f3f88939700 1 -- 172.21.15.71:0/1941368255 >> 172.21.15.71:0/1941368255 conn(0x7f3f840f1b40 msgr2=0x7f3f840f3f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:56:27.883 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:27.826+0000 7f3f88939700 1 -- 172.21.15.71:0/1941368255 shutdown_connections 2024-02-20T12:56:27.883 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:27.826+0000 7f3f88939700 1 -- 172.21.15.71:0/1941368255 wait complete. 2024-02-20T12:56:27.884 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:27.826+0000 7f3f88939700 1 Processor -- start 2024-02-20T12:56:27.884 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:27.827+0000 7f3f88939700 1 -- start start 2024-02-20T12:56:27.884 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:27.827+0000 7f3f88939700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3f840f63e0 0x7f3f8406ba80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:56:27.884 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:27.827+0000 7f3f88939700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f84069f00 con 0x7f3f840f63e0 2024-02-20T12:56:27.884 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:27.827+0000 7f3f837fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3f840f63e0 0x7f3f8406ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:56:27.884 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:27.827+0000 7f3f837fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3f840f63e0 0x7f3f8406ba80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.71:34414/0 (socket says 172.21.15.71:34414) 2024-02-20T12:56:27.884 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:27.827+0000 7f3f837fe700 1 -- 172.21.15.71:0/659865422 learned_addr learned my addr 172.21.15.71:0/659865422 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-02-20T12:56:27.884 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:27.827+0000 7f3f837fe700 1 -- 172.21.15.71:0/659865422 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f78005040 con 0x7f3f840f63e0 2024-02-20T12:56:27.884 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:27.827+0000 7f3f837fe700 1 --2- 172.21.15.71:0/659865422 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3f840f63e0 0x7f3f8406ba80 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 rx=0x7f3f780089f0 tx=0x7f3f7800b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:56:27.884 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:27.827+0000 7f3f6bfff700 1 -- 172.21.15.71:0/659865422 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3f78009320 con 0x7f3f840f63e0 2024-02-20T12:56:27.884 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:27.828+0000 7f3f88939700 1 -- 172.21.15.71:0/659865422 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f8406a0a0 con 0x7f3f840f63e0 2024-02-20T12:56:27.884 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:27.828+0000 7f3f6bfff700 1 -- 172.21.15.71:0/659865422 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(25 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f3f78013ec0 con 0x7f3f840f63e0 2024-02-20T12:56:27.884 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:27.828+0000 7f3f6bfff700 1 -- 172.21.15.71:0/659865422 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3f78014d30 con 0x7f3f840f63e0 2024-02-20T12:56:27.884 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:27.828+0000 7f3f88939700 1 -- 172.21.15.71:0/659865422 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f8406a530 con 0x7f3f840f63e0 2024-02-20T12:56:27.885 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:27.828+0000 7f3f6bfff700 1 -- 172.21.15.71:0/659865422 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 37151+0+0 (secure 0 0 0) 0x7f3f780095f0 con 0x7f3f840f63e0 2024-02-20T12:56:27.885 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:27.828+0000 7f3f88939700 1 -- 172.21.15.71:0/659865422 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f840591e0 con 0x7f3f840f63e0 2024-02-20T12:56:27.885 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:27.829+0000 7f3f6bfff700 1 --2- 172.21.15.71:0/659865422 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f3f6c0310e0 0x7f3f6c0335a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:56:27.885 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:27.829+0000 7f3f6bfff700 1 -- 172.21.15.71:0/659865422 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1065+0+0 (secure 0 0 0) 0x7f3f78042460 con 0x7f3f840f63e0 2024-02-20T12:56:27.885 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:27.829+0000 7f3f82ffd700 1 --2- 172.21.15.71:0/659865422 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f3f6c0310e0 0x7f3f6c0335a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:56:27.885 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:27.829+0000 7f3f82ffd700 1 --2- 172.21.15.71:0/659865422 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f3f6c0310e0 0x7f3f6c0335a0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 rx=0x7f3f740096a0 tx=0x7f3f74006b40).ready entity=mgr.14156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:56:27.885 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:27.834+0000 7f3f6bfff700 1 -- 172.21.15.71:0/659865422 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+153237 (secure 0 0 0) 0x7f3f7801d7f0 con 0x7f3f840f63e0 2024-02-20T12:56:28.039 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:28.039+0000 7f3f88939700 1 -- 172.21.15.71:0/659865422 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f3f840fbed0 con 0x7f3f840f63e0 2024-02-20T12:56:28.040 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:28.039+0000 7f3f6bfff700 1 -- 172.21.15.71:0/659865422 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7f3f7800fe40 con 0x7f3f840f63e0 2024-02-20T12:56:28.040 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-20T12:56:28.040 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":1,"fsid":"680ee126-cfee-11ee-95bb-87774f69a715","modified":"2024-02-20T12:49:44.699059Z","created":"2024-02-20T12:49:44.699059Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi064","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:3300","nonce":0},{"type":"v1","addr":"172.21.15.64:6789","nonce":0}]},"addr":"172.21.15.64:6789/0","public_addr":"172.21.15.64:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-20T12:56:28.041 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:28.041+0000 7f3f88939700 1 -- 172.21.15.71:0/659865422 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f3f6c0310e0 msgr2=0x7f3f6c0335a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:56:28.041 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:28.041+0000 7f3f88939700 1 --2- 172.21.15.71:0/659865422 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f3f6c0310e0 0x7f3f6c0335a0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 rx=0x7f3f740096a0 tx=0x7f3f74006b40).stop 2024-02-20T12:56:28.041 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:28.041+0000 7f3f88939700 1 -- 172.21.15.71:0/659865422 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3f840f63e0 msgr2=0x7f3f8406ba80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:56:28.041 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:28.041+0000 7f3f88939700 1 --2- 172.21.15.71:0/659865422 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3f840f63e0 0x7f3f8406ba80 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 rx=0x7f3f780089f0 tx=0x7f3f7800b040).stop 2024-02-20T12:56:28.042 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:28.041+0000 7f3f88939700 1 -- 172.21.15.71:0/659865422 shutdown_connections 2024-02-20T12:56:28.042 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:28.041+0000 7f3f88939700 1 --2- 172.21.15.71:0/659865422 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f3f6c0310e0 0x7f3f6c0335a0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:56:28.042 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:28.041+0000 7f3f88939700 1 --2- 172.21.15.71:0/659865422 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3f840f63e0 0x7f3f8406ba80 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:56:28.042 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:28.041+0000 7f3f88939700 1 -- 172.21.15.71:0/659865422 >> 172.21.15.71:0/659865422 conn(0x7f3f840f1b40 msgr2=0x7f3f840f2660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:56:28.042 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:28.041+0000 7f3f88939700 1 -- 172.21.15.71:0/659865422 shutdown_connections 2024-02-20T12:56:28.042 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:28.041+0000 7f3f88939700 1 -- 172.21.15.71:0/659865422 wait complete. 2024-02-20T12:56:28.042 INFO:teuthology.orchestra.run.smithi071.stderr:dumped monmap epoch 1 2024-02-20T12:56:28.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:56:28 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:56:28.040167+0000 mon.smithi064 (mon.0) 196 : audit [DBG] from='client.? 172.21.15.71:0/659865422' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-20T12:56:29.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:56:29 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:56:27.755441+0000 mgr.smithi064.vriyor (mgr.14156) 169 : cluster [DBG] pgmap v161: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:56:29.722 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-20T12:56:29.722 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/daemon-base:latest-pacific shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 680ee126-cfee-11ee-95bb-87774f69a715 -- ceph mon dump -f json 2024-02-20T12:56:31.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:56:31 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:56:29.755705+0000 mgr.smithi064.vriyor (mgr.14156) 170 : cluster [DBG] pgmap v162: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:56:31.718 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:31.715+0000 7f1acd196700 1 -- 172.21.15.71:0/1192085563 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1ac8060360 msgr2=0x7f1ac80f9190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:56:31.718 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:31.715+0000 7f1acd196700 1 --2- 172.21.15.71:0/1192085563 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1ac8060360 0x7f1ac80f9190 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 rx=0x7f1abc009fb0 tx=0x7f1abc009b30).stop 2024-02-20T12:56:31.718 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:31.715+0000 7f1acd196700 1 -- 172.21.15.71:0/1192085563 shutdown_connections 2024-02-20T12:56:31.718 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:31.715+0000 7f1acd196700 1 --2- 172.21.15.71:0/1192085563 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1ac8060360 0x7f1ac80f9190 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:56:31.718 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:31.715+0000 7f1acd196700 1 -- 172.21.15.71:0/1192085563 >> 172.21.15.71:0/1192085563 conn(0x7f1ac80f0c10 msgr2=0x7f1ac80f3030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:56:31.718 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:31.715+0000 7f1acd196700 1 -- 172.21.15.71:0/1192085563 shutdown_connections 2024-02-20T12:56:31.718 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:31.715+0000 7f1acd196700 1 -- 172.21.15.71:0/1192085563 wait complete. 2024-02-20T12:56:31.718 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:31.716+0000 7f1acd196700 1 Processor -- start 2024-02-20T12:56:31.718 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:31.716+0000 7f1acd196700 1 -- start start 2024-02-20T12:56:31.718 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:31.716+0000 7f1acd196700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1ac8060360 0x7f1ac8106790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:56:31.718 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:31.716+0000 7f1acd196700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ac8106c80 con 0x7f1ac8060360 2024-02-20T12:56:31.719 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:31.717+0000 7f1ac6d9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1ac8060360 0x7f1ac8106790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:56:31.719 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:31.717+0000 7f1ac6d9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1ac8060360 0x7f1ac8106790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.71:34432/0 (socket says 172.21.15.71:34432) 2024-02-20T12:56:31.719 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:31.717+0000 7f1ac6d9d700 1 -- 172.21.15.71:0/3394275614 learned_addr learned my addr 172.21.15.71:0/3394275614 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-02-20T12:56:31.719 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:31.717+0000 7f1ac6d9d700 1 -- 172.21.15.71:0/3394275614 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1abc005040 con 0x7f1ac8060360 2024-02-20T12:56:31.719 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:31.717+0000 7f1ac6d9d700 1 --2- 172.21.15.71:0/3394275614 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1ac8060360 0x7f1ac8106790 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 rx=0x7f1abc009f80 tx=0x7f1abc00b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:56:31.719 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:31.717+0000 7f1aaf7fe700 1 -- 172.21.15.71:0/3394275614 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1abc009320 con 0x7f1ac8060360 2024-02-20T12:56:31.719 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:31.717+0000 7f1aaf7fe700 1 -- 172.21.15.71:0/3394275614 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(25 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f1abc013680 con 0x7f1ac8060360 2024-02-20T12:56:31.719 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:31.717+0000 7f1acd196700 1 -- 172.21.15.71:0/3394275614 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ac8106e80 con 0x7f1ac8060360 2024-02-20T12:56:31.719 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:31.717+0000 7f1aaf7fe700 1 -- 172.21.15.71:0/3394275614 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1abc01eda0 con 0x7f1ac8060360 2024-02-20T12:56:31.719 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:31.717+0000 7f1acd196700 1 -- 172.21.15.71:0/3394275614 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ac8107310 con 0x7f1ac8060360 2024-02-20T12:56:31.720 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:31.718+0000 7f1aaf7fe700 1 -- 172.21.15.71:0/3394275614 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 37151+0+0 (secure 0 0 0) 0x7f1abc01e420 con 0x7f1ac8060360 2024-02-20T12:56:31.720 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:31.718+0000 7f1acd196700 1 -- 172.21.15.71:0/3394275614 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ac8045bb0 con 0x7f1ac8060360 2024-02-20T12:56:31.720 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:31.718+0000 7f1aaf7fe700 1 --2- 172.21.15.71:0/3394275614 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f1ab0031090 0x7f1ab0033550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:56:31.720 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:31.718+0000 7f1aaf7fe700 1 -- 172.21.15.71:0/3394275614 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1065+0+0 (secure 0 0 0) 0x7f1abc042360 con 0x7f1ac8060360 2024-02-20T12:56:31.720 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:31.718+0000 7f1ac659c700 1 --2- 172.21.15.71:0/3394275614 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f1ab0031090 0x7f1ab0033550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:56:31.720 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:31.719+0000 7f1ac659c700 1 --2- 172.21.15.71:0/3394275614 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f1ab0031090 0x7f1ab0033550 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 rx=0x7f1ab80096a0 tx=0x7f1ab8006b40).ready entity=mgr.14156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:56:31.723 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:31.723+0000 7f1aaf7fe700 1 -- 172.21.15.71:0/3394275614 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+153237 (secure 0 0 0) 0x7f1abc01d380 con 0x7f1ac8060360 2024-02-20T12:56:31.928 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:31.928+0000 7f1acd196700 1 -- 172.21.15.71:0/3394275614 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f1ac81081c0 con 0x7f1ac8060360 2024-02-20T12:56:31.929 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:31.929+0000 7f1aaf7fe700 1 -- 172.21.15.71:0/3394275614 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7f1abc004490 con 0x7f1ac8060360 2024-02-20T12:56:31.929 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-20T12:56:31.929 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":1,"fsid":"680ee126-cfee-11ee-95bb-87774f69a715","modified":"2024-02-20T12:49:44.699059Z","created":"2024-02-20T12:49:44.699059Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi064","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:3300","nonce":0},{"type":"v1","addr":"172.21.15.64:6789","nonce":0}]},"addr":"172.21.15.64:6789/0","public_addr":"172.21.15.64:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-20T12:56:31.931 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:31.930+0000 7f1acd196700 1 -- 172.21.15.71:0/3394275614 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f1ab0031090 msgr2=0x7f1ab0033550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:56:31.931 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:31.930+0000 7f1acd196700 1 --2- 172.21.15.71:0/3394275614 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f1ab0031090 0x7f1ab0033550 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 rx=0x7f1ab80096a0 tx=0x7f1ab8006b40).stop 2024-02-20T12:56:31.931 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:31.930+0000 7f1acd196700 1 -- 172.21.15.71:0/3394275614 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1ac8060360 msgr2=0x7f1ac8106790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:56:31.931 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:31.930+0000 7f1acd196700 1 --2- 172.21.15.71:0/3394275614 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1ac8060360 0x7f1ac8106790 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 rx=0x7f1abc009f80 tx=0x7f1abc00b040).stop 2024-02-20T12:56:31.931 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:31.930+0000 7f1acd196700 1 -- 172.21.15.71:0/3394275614 shutdown_connections 2024-02-20T12:56:31.931 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:31.930+0000 7f1acd196700 1 --2- 172.21.15.71:0/3394275614 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f1ab0031090 0x7f1ab0033550 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:56:31.931 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:31.930+0000 7f1acd196700 1 --2- 172.21.15.71:0/3394275614 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1ac8060360 0x7f1ac8106790 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:56:31.931 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:31.930+0000 7f1acd196700 1 -- 172.21.15.71:0/3394275614 >> 172.21.15.71:0/3394275614 conn(0x7f1ac80f0c10 msgr2=0x7f1ac80f16f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:56:31.932 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:31.930+0000 7f1acd196700 1 -- 172.21.15.71:0/3394275614 shutdown_connections 2024-02-20T12:56:31.932 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:31.930+0000 7f1acd196700 1 -- 172.21.15.71:0/3394275614 wait complete. 2024-02-20T12:56:31.932 INFO:teuthology.orchestra.run.smithi071.stderr:dumped monmap epoch 1 2024-02-20T12:56:32.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:56:32 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:56:31.929625+0000 mon.smithi064 (mon.0) 197 : audit [DBG] from='client.? 172.21.15.71:0/3394275614' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-20T12:56:33.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:56:33 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:56:31.756046+0000 mgr.smithi064.vriyor (mgr.14156) 171 : cluster [DBG] pgmap v163: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:56:33.697 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-20T12:56:33.697 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/daemon-base:latest-pacific shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 680ee126-cfee-11ee-95bb-87774f69a715 -- ceph mon dump -f json 2024-02-20T12:56:35.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:56:35 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:56:33.756360+0000 mgr.smithi064.vriyor (mgr.14156) 172 : cluster [DBG] pgmap v164: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:56:35.662 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:35.660+0000 7feb2b1af700 1 -- 172.21.15.71:0/2350622205 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feb240f54b0 msgr2=0x7feb240f58d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:56:35.662 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:35.660+0000 7feb2b1af700 1 --2- 172.21.15.71:0/2350622205 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feb240f54b0 0x7feb240f58d0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 rx=0x7feb18009fb0 tx=0x7feb18009b30).stop 2024-02-20T12:56:35.662 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:35.660+0000 7feb2b1af700 1 -- 172.21.15.71:0/2350622205 shutdown_connections 2024-02-20T12:56:35.662 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:35.660+0000 7feb2b1af700 1 --2- 172.21.15.71:0/2350622205 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feb240f54b0 0x7feb240f58d0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:56:35.662 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:35.660+0000 7feb2b1af700 1 -- 172.21.15.71:0/2350622205 >> 172.21.15.71:0/2350622205 conn(0x7feb240f0c10 msgr2=0x7feb240f3030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:56:35.662 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:35.660+0000 7feb2b1af700 1 -- 172.21.15.71:0/2350622205 shutdown_connections 2024-02-20T12:56:35.662 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:35.660+0000 7feb2b1af700 1 -- 172.21.15.71:0/2350622205 wait complete. 2024-02-20T12:56:35.662 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:35.660+0000 7feb2b1af700 1 Processor -- start 2024-02-20T12:56:35.662 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:35.660+0000 7feb2b1af700 1 -- start start 2024-02-20T12:56:35.662 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:35.661+0000 7feb2b1af700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feb240f54b0 0x7feb240fd490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:56:35.663 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:35.661+0000 7feb2b1af700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb240fb910 con 0x7feb240f54b0 2024-02-20T12:56:35.665 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:35.661+0000 7feb28f4b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feb240f54b0 0x7feb240fd490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:56:35.665 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:35.661+0000 7feb28f4b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feb240f54b0 0x7feb240fd490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.71:34448/0 (socket says 172.21.15.71:34448) 2024-02-20T12:56:35.665 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:35.661+0000 7feb28f4b700 1 -- 172.21.15.71:0/2745745091 learned_addr learned my addr 172.21.15.71:0/2745745091 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-02-20T12:56:35.665 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:35.661+0000 7feb28f4b700 1 -- 172.21.15.71:0/2745745091 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb18005040 con 0x7feb240f54b0 2024-02-20T12:56:35.665 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:35.661+0000 7feb28f4b700 1 --2- 172.21.15.71:0/2745745091 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feb240f54b0 0x7feb240fd490 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 rx=0x7feb18009f80 tx=0x7feb1800b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:56:35.665 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:35.662+0000 7feb217fa700 1 -- 172.21.15.71:0/2745745091 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7feb18009320 con 0x7feb240f54b0 2024-02-20T12:56:35.665 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:35.662+0000 7feb2b1af700 1 -- 172.21.15.71:0/2745745091 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb240fbab0 con 0x7feb240f54b0 2024-02-20T12:56:35.665 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:35.662+0000 7feb217fa700 1 -- 172.21.15.71:0/2745745091 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(25 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7feb18013380 con 0x7feb240f54b0 2024-02-20T12:56:35.665 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:35.662+0000 7feb217fa700 1 -- 172.21.15.71:0/2745745091 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7feb1801ebd0 con 0x7feb240f54b0 2024-02-20T12:56:35.665 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:35.662+0000 7feb2b1af700 1 -- 172.21.15.71:0/2745745091 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb240fbf40 con 0x7feb240f54b0 2024-02-20T12:56:35.665 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:35.662+0000 7feb217fa700 1 -- 172.21.15.71:0/2745745091 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 37151+0+0 (secure 0 0 0) 0x7feb1801e420 con 0x7feb240f54b0 2024-02-20T12:56:35.665 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:35.663+0000 7feb2b1af700 1 -- 172.21.15.71:0/2745745091 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb240591e0 con 0x7feb240f54b0 2024-02-20T12:56:35.666 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:35.663+0000 7feb217fa700 1 --2- 172.21.15.71:0/2745745091 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7feb14031090 0x7feb14033550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:56:35.666 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:35.663+0000 7feb217fa700 1 -- 172.21.15.71:0/2745745091 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1065+0+0 (secure 0 0 0) 0x7feb18042220 con 0x7feb240f54b0 2024-02-20T12:56:35.666 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:35.663+0000 7feb23fff700 1 --2- 172.21.15.71:0/2745745091 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7feb14031090 0x7feb14033550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:56:35.666 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:35.663+0000 7feb23fff700 1 --2- 172.21.15.71:0/2745745091 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7feb14031090 0x7feb14033550 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 rx=0x7feb100096a0 tx=0x7feb10006b40).ready entity=mgr.14156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:56:35.669 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:35.667+0000 7feb217fa700 1 -- 172.21.15.71:0/2745745091 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+153237 (secure 0 0 0) 0x7feb1801d380 con 0x7feb240f54b0 2024-02-20T12:56:35.874 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:35.873+0000 7feb2b1af700 1 -- 172.21.15.71:0/2745745091 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7feb240fafa0 con 0x7feb240f54b0 2024-02-20T12:56:35.875 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:35.874+0000 7feb217fa700 1 -- 172.21.15.71:0/2745745091 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7feb1801cb10 con 0x7feb240f54b0 2024-02-20T12:56:35.875 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-20T12:56:35.875 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":1,"fsid":"680ee126-cfee-11ee-95bb-87774f69a715","modified":"2024-02-20T12:49:44.699059Z","created":"2024-02-20T12:49:44.699059Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi064","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:3300","nonce":0},{"type":"v1","addr":"172.21.15.64:6789","nonce":0}]},"addr":"172.21.15.64:6789/0","public_addr":"172.21.15.64:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-20T12:56:35.877 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:35.875+0000 7feb2b1af700 1 -- 172.21.15.71:0/2745745091 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7feb14031090 msgr2=0x7feb14033550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:56:35.877 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:35.875+0000 7feb2b1af700 1 --2- 172.21.15.71:0/2745745091 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7feb14031090 0x7feb14033550 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 rx=0x7feb100096a0 tx=0x7feb10006b40).stop 2024-02-20T12:56:35.877 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:35.875+0000 7feb2b1af700 1 -- 172.21.15.71:0/2745745091 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feb240f54b0 msgr2=0x7feb240fd490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:56:35.877 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:35.875+0000 7feb2b1af700 1 --2- 172.21.15.71:0/2745745091 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feb240f54b0 0x7feb240fd490 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 rx=0x7feb18009f80 tx=0x7feb1800b040).stop 2024-02-20T12:56:35.877 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:35.875+0000 7feb2b1af700 1 -- 172.21.15.71:0/2745745091 shutdown_connections 2024-02-20T12:56:35.877 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:35.875+0000 7feb2b1af700 1 --2- 172.21.15.71:0/2745745091 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7feb14031090 0x7feb14033550 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:56:35.877 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:35.875+0000 7feb2b1af700 1 --2- 172.21.15.71:0/2745745091 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feb240f54b0 0x7feb240fd490 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:56:35.877 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:35.875+0000 7feb2b1af700 1 -- 172.21.15.71:0/2745745091 >> 172.21.15.71:0/2745745091 conn(0x7feb240f0c10 msgr2=0x7feb2405ff20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:56:35.877 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:35.875+0000 7feb2b1af700 1 -- 172.21.15.71:0/2745745091 shutdown_connections 2024-02-20T12:56:35.877 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:35.875+0000 7feb2b1af700 1 -- 172.21.15.71:0/2745745091 wait complete. 2024-02-20T12:56:35.878 INFO:teuthology.orchestra.run.smithi071.stderr:dumped monmap epoch 1 2024-02-20T12:56:36.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:56:36 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:56:35.875586+0000 mon.smithi064 (mon.0) 198 : audit [DBG] from='client.? 172.21.15.71:0/2745745091' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-20T12:56:37.471 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-20T12:56:37.472 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/daemon-base:latest-pacific shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 680ee126-cfee-11ee-95bb-87774f69a715 -- ceph mon dump -f json 2024-02-20T12:56:37.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:56:37 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:56:35.756664+0000 mgr.smithi064.vriyor (mgr.14156) 173 : cluster [DBG] pgmap v165: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:56:39.456 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:39.453+0000 7f051e9a0700 1 -- 172.21.15.71:0/3267860012 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05180602c0 msgr2=0x7f05180f9050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:56:39.456 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:39.453+0000 7f051e9a0700 1 --2- 172.21.15.71:0/3267860012 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05180602c0 0x7f05180f9050 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 rx=0x7f0508009fb0 tx=0x7f0508009b30).stop 2024-02-20T12:56:39.456 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:39.453+0000 7f051e9a0700 1 -- 172.21.15.71:0/3267860012 shutdown_connections 2024-02-20T12:56:39.456 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:39.453+0000 7f051e9a0700 1 --2- 172.21.15.71:0/3267860012 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05180602c0 0x7f05180f9050 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:56:39.456 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:39.453+0000 7f051e9a0700 1 -- 172.21.15.71:0/3267860012 >> 172.21.15.71:0/3267860012 conn(0x7f05180f0a30 msgr2=0x7f05180f2e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:56:39.456 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:39.453+0000 7f051e9a0700 1 -- 172.21.15.71:0/3267860012 shutdown_connections 2024-02-20T12:56:39.456 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:39.453+0000 7f051e9a0700 1 -- 172.21.15.71:0/3267860012 wait complete. 2024-02-20T12:56:39.456 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:39.453+0000 7f051e9a0700 1 Processor -- start 2024-02-20T12:56:39.457 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:39.454+0000 7f051e9a0700 1 -- start start 2024-02-20T12:56:39.457 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:39.454+0000 7f051e9a0700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05180602c0 0x7f05180f6ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:56:39.457 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:39.454+0000 7f051e9a0700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05180f4f50 con 0x7f05180602c0 2024-02-20T12:56:39.457 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:39.454+0000 7f051d99e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05180602c0 0x7f05180f6ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:56:39.457 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:39.454+0000 7f051d99e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05180602c0 0x7f05180f6ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.71:38104/0 (socket says 172.21.15.71:38104) 2024-02-20T12:56:39.457 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:39.454+0000 7f051d99e700 1 -- 172.21.15.71:0/3601438834 learned_addr learned my addr 172.21.15.71:0/3601438834 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-02-20T12:56:39.457 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:39.454+0000 7f051d99e700 1 -- 172.21.15.71:0/3601438834 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0508005040 con 0x7f05180602c0 2024-02-20T12:56:39.457 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:39.454+0000 7f051d99e700 1 --2- 172.21.15.71:0/3601438834 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05180602c0 0x7f05180f6ad0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 rx=0x7f0508009d00 tx=0x7f0508009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:56:39.457 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:39.454+0000 7f050e7fc700 1 -- 172.21.15.71:0/3601438834 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0508018070 con 0x7f05180602c0 2024-02-20T12:56:39.457 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:39.454+0000 7f051e9a0700 1 -- 172.21.15.71:0/3601438834 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05180f50f0 con 0x7f05180602c0 2024-02-20T12:56:39.457 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:39.455+0000 7f051e9a0700 1 -- 172.21.15.71:0/3601438834 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05180f5580 con 0x7f05180602c0 2024-02-20T12:56:39.457 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:39.455+0000 7f050e7fc700 1 -- 172.21.15.71:0/3601438834 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(25 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f050801d420 con 0x7f05180602c0 2024-02-20T12:56:39.457 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:39.455+0000 7f050e7fc700 1 -- 172.21.15.71:0/3601438834 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f050801c3c0 con 0x7f05180602c0 2024-02-20T12:56:39.458 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:39.455+0000 7f050e7fc700 1 -- 172.21.15.71:0/3601438834 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 37151+0+0 (secure 0 0 0) 0x7f050801d950 con 0x7f05180602c0 2024-02-20T12:56:39.458 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:39.455+0000 7f051e9a0700 1 -- 172.21.15.71:0/3601438834 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05180591e0 con 0x7f05180602c0 2024-02-20T12:56:39.458 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:39.455+0000 7f050e7fc700 1 --2- 172.21.15.71:0/3601438834 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f0504031090 0x7f0504033550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:56:39.458 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:39.455+0000 7f050e7fc700 1 -- 172.21.15.71:0/3601438834 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1065+0+0 (secure 0 0 0) 0x7f0508042980 con 0x7f05180602c0 2024-02-20T12:56:39.459 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:39.456+0000 7f051d19d700 1 --2- 172.21.15.71:0/3601438834 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f0504031090 0x7f0504033550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:56:39.459 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:39.456+0000 7f051d19d700 1 --2- 172.21.15.71:0/3601438834 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f0504031090 0x7f0504033550 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 rx=0x7f05140096a0 tx=0x7f0514006b40).ready entity=mgr.14156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:56:39.462 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:39.461+0000 7f050e7fc700 1 -- 172.21.15.71:0/3601438834 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+153237 (secure 0 0 0) 0x7f050801b370 con 0x7f05180602c0 2024-02-20T12:56:39.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:56:39 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:56:37.757074+0000 mgr.smithi064.vriyor (mgr.14156) 174 : cluster [DBG] pgmap v166: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:56:39.668 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:39.666+0000 7f051e9a0700 1 -- 172.21.15.71:0/3601438834 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f05180fbec0 con 0x7f05180602c0 2024-02-20T12:56:39.668 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:39.667+0000 7f050e7fc700 1 -- 172.21.15.71:0/3601438834 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7f0508025b80 con 0x7f05180602c0 2024-02-20T12:56:39.668 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-20T12:56:39.668 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":1,"fsid":"680ee126-cfee-11ee-95bb-87774f69a715","modified":"2024-02-20T12:49:44.699059Z","created":"2024-02-20T12:49:44.699059Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi064","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:3300","nonce":0},{"type":"v1","addr":"172.21.15.64:6789","nonce":0}]},"addr":"172.21.15.64:6789/0","public_addr":"172.21.15.64:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-20T12:56:39.670 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:39.668+0000 7f051e9a0700 1 -- 172.21.15.71:0/3601438834 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f0504031090 msgr2=0x7f0504033550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:56:39.670 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:39.668+0000 7f051e9a0700 1 --2- 172.21.15.71:0/3601438834 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f0504031090 0x7f0504033550 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 rx=0x7f05140096a0 tx=0x7f0514006b40).stop 2024-02-20T12:56:39.670 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:39.668+0000 7f051e9a0700 1 -- 172.21.15.71:0/3601438834 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05180602c0 msgr2=0x7f05180f6ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:56:39.670 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:39.668+0000 7f051e9a0700 1 --2- 172.21.15.71:0/3601438834 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05180602c0 0x7f05180f6ad0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 rx=0x7f0508009d00 tx=0x7f0508009b30).stop 2024-02-20T12:56:39.670 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:39.668+0000 7f051e9a0700 1 -- 172.21.15.71:0/3601438834 shutdown_connections 2024-02-20T12:56:39.670 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:39.668+0000 7f051e9a0700 1 --2- 172.21.15.71:0/3601438834 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f0504031090 0x7f0504033550 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:56:39.670 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:39.668+0000 7f051e9a0700 1 --2- 172.21.15.71:0/3601438834 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05180602c0 0x7f05180f6ad0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:56:39.670 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:39.668+0000 7f051e9a0700 1 -- 172.21.15.71:0/3601438834 >> 172.21.15.71:0/3601438834 conn(0x7f05180f0a30 msgr2=0x7f05180f1520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:56:39.670 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:39.668+0000 7f051e9a0700 1 -- 172.21.15.71:0/3601438834 shutdown_connections 2024-02-20T12:56:39.670 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:39.668+0000 7f051e9a0700 1 -- 172.21.15.71:0/3601438834 wait complete. 2024-02-20T12:56:39.671 INFO:teuthology.orchestra.run.smithi071.stderr:dumped monmap epoch 1 2024-02-20T12:56:40.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:56:40 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:56:39.668676+0000 mon.smithi064 (mon.0) 199 : audit [DBG] from='client.? 172.21.15.71:0/3601438834' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-20T12:56:41.460 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-20T12:56:41.460 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/daemon-base:latest-pacific shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 680ee126-cfee-11ee-95bb-87774f69a715 -- ceph mon dump -f json 2024-02-20T12:56:41.521 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:56:41 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:56:39.757412+0000 mgr.smithi064.vriyor (mgr.14156) 175 : cluster [DBG] pgmap v167: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:56:41.521 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:56:41 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:56:40.918270+0000 mon.smithi064 (mon.0) 200 : audit [INF] from='mgr.14156 172.21.15.64:0/3284974785' entity='mgr.smithi064.vriyor' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi064", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-02-20T12:56:41.521 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:56:41 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:56:40.920849+0000 mon.smithi064 (mon.0) 201 : audit [INF] from='mgr.14156 172.21.15.64:0/3284974785' entity='mgr.smithi064.vriyor' cmd='[{"prefix": "auth get-or-create", "entity": "client.crash.smithi064", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]': finished 2024-02-20T12:56:41.521 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:56:41 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:56:40.921768+0000 mon.smithi064 (mon.0) 202 : audit [DBG] from='mgr.14156 172.21.15.64:0/3284974785' entity='mgr.smithi064.vriyor' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-20T12:56:42.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:56:42 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cephadm 2024-02-20T12:56:40.915795+0000 mgr.smithi064.vriyor (mgr.14156) 176 : cephadm [ERR] cephadm exited with an error code: 1, stderr:Non-zero exit code 125 from /bin/podman container inspect --format {{.State.Status}} ceph-680ee126-cfee-11ee-95bb-87774f69a715-alertmanager.smithi064 2024-02-20T12:56:42.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:56:42 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: /bin/podman: stderr Error: no such container ceph-680ee126-cfee-11ee-95bb-87774f69a715-alertmanager.smithi064 2024-02-20T12:56:42.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:56:42 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: Deploy daemon alertmanager.smithi064 ... 2024-02-20T12:56:42.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:56:42 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: Non-zero exit code 125 from /bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint stat --init -e CONTAINER_IMAGE=docker.io/prom/alertmanager:v0.20.0 -e NODE_NAME=smithi064 -e CEPH_USE_RANDOM_NONCE=1 docker.io/prom/alertmanager:v0.20.0 -c %u %g /etc/alertmanager 2024-02-20T12:56:42.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:56:42 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: stat: stderr Trying to pull docker.io/prom/alertmanager:v0.20.0... 2024-02-20T12:56:42.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:56:42 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: stat: stderr time="2024-02-20T12:52:19Z" level=warning msg="Failed, retrying in 1s ... (1/3). Error: copying system image from manifest list: determining manifest MIME type for docker://prom/alertmanager:v0.20.0: reading manifest sha256:b9323917a2eda265bec69e59a457f001c529facbbc8166df277f4850cdac61a0 in docker.io/prom/alertmanager: toomanyrequests: You have reached your pull rate limit. You may increase the limit by authenticating and upgrading: https://www.docker.com/increase-rate-limit" 2024-02-20T12:56:42.563 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:56:42 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: stat: stderr time="2024-02-20T12:52:56Z" level=warning msg="Failed, retrying in 1s ... (2/3). Error: initializing source docker://prom/alertmanager:v0.20.0: (Mirrors also failed: [docker-mirror.front.sepia.ceph.com:5000/prom/alertmanager:v0.20.0: pinging container registry docker-mirror.front.sepia.ceph.com:5000: Get \"https://docker-mirror.front.sepia.ceph.com:5000/v2/\": dial tcp 172.21.0.79:5000: connect: no route to host]): docker.io/prom/alertmanager:v0.20.0: reading manifest v0.20.0 in docker.io/prom/alertmanager: toomanyrequests: You have reached your pull rate limit. You may increase the limit by authenticating and upgrading: https://www.docker.com/increase-rate-limit" 2024-02-20T12:56:42.563 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:56:42 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: stat: stderr time="2024-02-20T12:53:34Z" level=warning msg="Failed, retrying in 1s ... (3/3). Error: initializing source docker://prom/alertmanager:v0.20.0: (Mirrors also failed: [docker-mirror.front.sepia.ceph.com:5000/prom/alertmanager:v0.20.0: pinging container registry docker-mirror.front.sepia.ceph.com:5000: Get \"https://docker-mirror.front.sepia.ceph.com:5000/v2/\": dial tcp 172.21.0.79:5000: connect: no route to host]): docker.io/prom/alertmanager:v0.20.0: reading manifest v0.20.0 in docker.io/prom/alertmanager: toomanyrequests: You have reached your pull rate limit. You may increase the limit by authenticating and upgrading: https://www.docker.com/increase-rate-limit" 2024-02-20T12:56:42.563 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:56:42 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: stat: stderr Error: copying system image from manifest list: determining manifest MIME type for docker://prom/alertmanager:v0.20.0: reading manifest sha256:b9323917a2eda265bec69e59a457f001c529facbbc8166df277f4850cdac61a0 in docker.io/prom/alertmanager: toomanyrequests: You have reached your pull rate limit. You may increase the limit by authenticating and upgrading: https://www.docker.com/increase-rate-limit 2024-02-20T12:56:42.564 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:56:42 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: Non-zero exit code 125 from /bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint stat --init -e CONTAINER_IMAGE=docker.io/prom/alertmanager:v0.20.0 -e NODE_NAME=smithi064 -e CEPH_USE_RANDOM_NONCE=1 docker.io/prom/alertmanager:v0.20.0 -c %u %g /etc/prometheus 2024-02-20T12:56:42.564 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:56:42 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: stat: stderr Trying to pull docker.io/prom/alertmanager:v0.20.0... 2024-02-20T12:56:42.564 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:56:42 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: stat: stderr time="2024-02-20T12:54:48Z" level=warning msg="Failed, retrying in 1s ... (1/3). Error: initializing source docker://prom/alertmanager:v0.20.0: (Mirrors also failed: [docker-mirror.front.sepia.ceph.com:5000/prom/alertmanager:v0.20.0: pinging container registry docker-mirror.front.sepia.ceph.com:5000: Get \"https://docker-mirror.front.sepia.ceph.com:5000/v2/\": dial tcp 172.21.0.79:5000: connect: no route to host]): docker.io/prom/alertmanager:v0.20.0: reading manifest v0.20.0 in docker.io/prom/alertmanager: toomanyrequests: You have reached your pull rate limit. You may increase the limit by authenticating and upgrading: https://www.docker.com/increase-rate-limit" 2024-02-20T12:56:42.564 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:56:42 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: stat: stderr time="2024-02-20T12:55:25Z" level=warning msg="Failed, retrying in 1s ... (2/3). Error: initializing source docker://prom/alertmanager:v0.20.0: (Mirrors also failed: [docker-mirror.front.sepia.ceph.com:5000/prom/alertmanager:v0.20.0: pinging container registry docker-mirror.front.sepia.ceph.com:5000: Get \"https://docker-mirror.front.sepia.ceph.com:5000/v2/\": dial tcp 172.21.0.79:5000: connect: no route to host]): docker.io/prom/alertmanager:v0.20.0: reading manifest v0.20.0 in docker.io/prom/alertmanager: toomanyrequests: You have reached your pull rate limit. You may increase the limit by authenticating and upgrading: https://www.docker.com/increase-rate-limit" 2024-02-20T12:56:42.564 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:56:42 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: stat: stderr time="2024-02-20T12:56:03Z" level=warning msg="Failed, retrying in 1s ... (3/3). Error: initializing source docker://prom/alertmanager:v0.20.0: (Mirrors also failed: [docker-mirror.front.sepia.ceph.com:5000/prom/alertmanager:v0.20.0: pinging container registry docker-mirror.front.sepia.ceph.com:5000: Get \"https://docker-mirror.front.sepia.ceph.com:5000/v2/\": dial tcp 172.21.0.79:5000: connect: no route to host]): docker.io/prom/alertmanager:v0.20.0: reading manifest v0.20.0 in docker.io/prom/alertmanager: toomanyrequests: You have reached your pull rate limit. You may increase the limit by authenticating and upgrading: https://www.docker.com/increase-rate-limit" 2024-02-20T12:56:42.564 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:56:42 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: stat: stderr Error: initializing source docker://prom/alertmanager:v0.20.0: (Mirrors also failed: [docker-mirror.front.sepia.ceph.com:5000/prom/alertmanager:v0.20.0: pinging container registry docker-mirror.front.sepia.ceph.com:5000: Get "https://docker-mirror.front.sepia.ceph.com:5000/v2/": dial tcp 172.21.0.79:5000: connect: no route to host]): docker.io/prom/alertmanager:v0.20.0: reading manifest v0.20.0 in docker.io/prom/alertmanager: toomanyrequests: You have reached your pull rate limit. You may increase the limit by authenticating and upgrading: https://www.docker.com/increase-rate-limit 2024-02-20T12:56:42.564 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:56:42 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: Traceback (most recent call last): 2024-02-20T12:56:42.565 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:56:42 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: File "/var/lib/ceph/680ee126-cfee-11ee-95bb-87774f69a715/cephadm.d4237e4639c108308fe13147b1c08af93c3d5724d9ff21ae797eb4b78fea3931", line 8230, in 2024-02-20T12:56:42.565 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:56:42 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: main() 2024-02-20T12:56:42.565 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:56:42 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: File "/var/lib/ceph/680ee126-cfee-11ee-95bb-87774f69a715/cephadm.d4237e4639c108308fe13147b1c08af93c3d5724d9ff21ae797eb4b78fea3931", line 8218, in main 2024-02-20T12:56:42.565 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:56:42 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: r = ctx.func(ctx) 2024-02-20T12:56:42.565 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:56:42 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: File "/var/lib/ceph/680ee126-cfee-11ee-95bb-87774f69a715/cephadm.d4237e4639c108308fe13147b1c08af93c3d5724d9ff21ae797eb4b78fea3931", line 1759, in _default_image 2024-02-20T12:56:42.565 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:56:42 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: return func(ctx) 2024-02-20T12:56:42.565 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:56:42 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: File "/var/lib/ceph/680ee126-cfee-11ee-95bb-87774f69a715/cephadm.d4237e4639c108308fe13147b1c08af93c3d5724d9ff21ae797eb4b78fea3931", line 4344, in command_deploy 2024-02-20T12:56:42.565 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:56:42 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: uid, gid = extract_uid_gid_monitoring(ctx, daemon_type) 2024-02-20T12:56:42.565 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:56:42 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: File "/var/lib/ceph/680ee126-cfee-11ee-95bb-87774f69a715/cephadm.d4237e4639c108308fe13147b1c08af93c3d5724d9ff21ae797eb4b78fea3931", line 4275, in extract_uid_gid_monitoring 2024-02-20T12:56:42.565 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:56:42 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: uid, gid = extract_uid_gid(ctx, file_path=['/etc/alertmanager', '/etc/prometheus']) 2024-02-20T12:56:42.565 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:56:42 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: File "/var/lib/ceph/680ee126-cfee-11ee-95bb-87774f69a715/cephadm.d4237e4639c108308fe13147b1c08af93c3d5724d9ff21ae797eb4b78fea3931", line 2548, in extract_uid_gid 2024-02-20T12:56:42.565 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:56:42 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: raise RuntimeError('uid/gid not found') 2024-02-20T12:56:42.565 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:56:42 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: RuntimeError: uid/gid not found 2024-02-20T12:56:42.566 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:56:42 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: Traceback (most recent call last): 2024-02-20T12:56:42.566 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:56:42 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: File "/usr/share/ceph/mgr/cephadm/serve.py", line 1347, in _remote_connection 2024-02-20T12:56:42.566 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:56:42 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: yield (conn, connr) 2024-02-20T12:56:42.566 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:56:42 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: File "/usr/share/ceph/mgr/cephadm/serve.py", line 1244, in _run_cephadm 2024-02-20T12:56:42.566 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:56:42 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: code, '\n'.join(err))) 2024-02-20T12:56:42.566 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:56:42 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: orchestrator._interface.OrchestratorError: cephadm exited with an error code: 1, stderr:Non-zero exit code 125 from /bin/podman container inspect --format {{.State.Status}} ceph-680ee126-cfee-11ee-95bb-87774f69a715-alertmanager.smithi064 2024-02-20T12:56:42.566 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:56:42 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: /bin/podman: stderr Error: no such container ceph-680ee126-cfee-11ee-95bb-87774f69a715-alertmanager.smithi064 2024-02-20T12:56:42.566 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:56:42 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: Deploy daemon alertmanager.smithi064 ... 2024-02-20T12:56:42.566 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:56:42 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: Non-zero exit code 125 from /bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint stat --init -e CONTAINER_IMAGE=docker.io/prom/alertmanager:v0.20.0 -e NODE_NAME=smithi064 -e CEPH_USE_RANDOM_NONCE=1 docker.io/prom/alertmanager:v0.20.0 -c %u %g /etc/alertmanager 2024-02-20T12:56:42.566 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:56:42 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: stat: stderr Trying to pull docker.io/prom/alertmanager:v0.20.0... 2024-02-20T12:56:42.566 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:56:42 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: stat: stderr time="2024-02-20T12:52:19Z" level=warning msg="Failed, retrying in 1s ... (1/3). Error: copying system image from manifest list: determining manifest MIME type for docker://prom/alertmanager:v0.20.0: reading manifest sha256:b9323917a2eda265bec69e59a457f001c529facbbc8166df277f4850cdac61a0 in docker.io/prom/alertmanager: toomanyrequests: You have reached your pull rate limit. You may increase the limit by authenticating and upgrading: https://www.docker.com/increase-rate-limit" 2024-02-20T12:56:42.566 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:56:42 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: stat: stderr time="2024-02-20T12:52:56Z" level=warning msg="Failed, retrying in 1s ... (2/3). Error: initializing source docker://prom/alertmanager:v0.20.0: (Mirrors also failed: [docker-mirror.front.sepia.ceph.com:5000/prom/alertmanager:v0.20.0: pinging container registry docker-mirror.front.sepia.ceph.com:5000: Get \"https://docker-mirror.front.sepia.ceph.com:5000/v2/\": dial tcp 172.21.0.79:5000: connect: no route to host]): docker.io/prom/alertmanager:v0.20.0: reading manifest v0.20.0 in docker.io/prom/alertmanager: toomanyrequests: You have reached your pull rate limit. You may increase the limit by authenticating and upgrading: https://www.docker.com/increase-rate-limit" 2024-02-20T12:56:42.566 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:56:42 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: stat: stderr time="2024-02-20T12:53:34Z" level=warning msg="Failed, retrying in 1s ... (3/3). Error: initializing source docker://prom/alertmanager:v0.20.0: (Mirrors also failed: [docker-mirror.front.sepia.ceph.com:5000/prom/alertmanager:v0.20.0: pinging container registry docker-mirror.front.sepia.ceph.com:5000: Get \"https://docker-mirror.front.sepia.ceph.com:5000/v2/\": dial tcp 172.21.0.79:5000: connect: no route to host]): docker.io/prom/alertmanager:v0.20.0: reading manifest v0.20.0 in docker.io/prom/alertmanager: toomanyrequests: You have reached your pull rate limit. You may increase the limit by authenticating and upgrading: https://www.docker.com/increase-rate-limit" 2024-02-20T12:56:42.566 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:56:42 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: stat: stderr Error: copying system image from manifest list: determining manifest MIME type for docker://prom/alertmanager:v0.20.0: reading manifest sha256:b9323917a2eda265bec69e59a457f001c529facbbc8166df277f4850cdac61a0 in docker.io/prom/alertmanager: toomanyrequests: You have reached your pull rate limit. You may increase the limit by authenticating and upgrading: https://www.docker.com/increase-rate-limit 2024-02-20T12:56:42.567 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:56:42 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: Non-zero exit code 125 from /bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint stat --init -e CONTAINER_IMAGE=docker.io/prom/alertmanager:v0.20.0 -e NODE_NAME=smithi064 -e CEPH_USE_RANDOM_NONCE=1 docker.io/prom/alertmanager:v0.20.0 -c %u %g /etc/prometheus 2024-02-20T12:56:42.567 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:56:42 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: stat: stderr Trying to pull docker.io/prom/alertmanager:v0.20.0... 2024-02-20T12:56:42.567 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:56:42 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: stat: stderr time="2024-02-20T12:54:48Z" level=warning msg="Failed, retrying in 1s ... (1/3). Error: initializing source docker://prom/alertmanager:v0.20.0: (Mirrors also failed: [docker-mirror.front.sepia.ceph.com:5000/prom/alertmanager:v0.20.0: pinging container registry docker-mirror.front.sepia.ceph.com:5000: Get \"https://docker-mirror.front.sepia.ceph.com:5000/v2/\": dial tcp 172.21.0.79:5000: connect: no route to host]): docker.io/prom/alertmanager:v0.20.0: reading manifest v0.20.0 in docker.io/prom/alertmanager: toomanyrequests: You have reached your pull rate limit. You may increase the limit by authenticating and upgrading: https://www.docker.com/increase-rate-limit" 2024-02-20T12:56:42.567 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:56:42 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: stat: stderr time="2024-02-20T12:55:25Z" level=warning msg="Failed, retrying in 1s ... (2/3). Error: initializing source docker://prom/alertmanager:v0.20.0: (Mirrors also failed: [docker-mirror.front.sepia.ceph.com:5000/prom/alertmanager:v0.20.0: pinging container registry docker-mirror.front.sepia.ceph.com:5000: Get \"https://docker-mirror.front.sepia.ceph.com:5000/v2/\": dial tcp 172.21.0.79:5000: connect: no route to host]): docker.io/prom/alertmanager:v0.20.0: reading manifest v0.20.0 in docker.io/prom/alertmanager: toomanyrequests: You have reached your pull rate limit. You may increase the limit by authenticating and upgrading: https://www.docker.com/increase-rate-limit" 2024-02-20T12:56:42.567 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:56:42 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: stat: stderr time="2024-02-20T12:56:03Z" level=warning msg="Failed, retrying in 1s ... (3/3). Error: initializing source docker://prom/alertmanager:v0.20.0: (Mirrors also failed: [docker-mirror.front.sepia.ceph.com:5000/prom/alertmanager:v0.20.0: pinging container registry docker-mirror.front.sepia.ceph.com:5000: Get \"https://docker-mirror.front.sepia.ceph.com:5000/v2/\": dial tcp 172.21.0.79:5000: connect: no route to host]): docker.io/prom/alertmanager:v0.20.0: reading manifest v0.20.0 in docker.io/prom/alertmanager: toomanyrequests: You have reached your pull rate limit. You may increase the limit by authenticating and upgrading: https://www.docker.com/increase-rate-limit" 2024-02-20T12:56:42.567 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:56:42 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: stat: stderr Error: initializing source docker://prom/alertmanager:v0.20.0: (Mirrors also failed: [docker-mirror.front.sepia.ceph.com:5000/prom/alertmanager:v0.20.0: pinging container registry docker-mirror.front.sepia.ceph.com:5000: Get "https://docker-mirror.front.sepia.ceph.com:5000/v2/": dial tcp 172.21.0.79:5000: connect: no route to host]): docker.io/prom/alertmanager:v0.20.0: reading manifest v0.20.0 in docker.io/prom/alertmanager: toomanyrequests: You have reached your pull rate limit. You may increase the limit by authenticating and upgrading: https://www.docker.com/increase-rate-limit 2024-02-20T12:56:42.567 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:56:42 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: Traceback (most recent call last): 2024-02-20T12:56:42.567 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:56:42 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: File "/var/lib/ceph/680ee126-cfee-11ee-95bb-87774f69a715/cephadm.d4237e4639c108308fe13147b1c08af93c3d5724d9ff21ae797eb4b78fea3931", line 8230, in 2024-02-20T12:56:42.567 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:56:42 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: main() 2024-02-20T12:56:42.567 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:56:42 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: File "/var/lib/ceph/680ee126-cfee-11ee-95bb-87774f69a715/cephadm.d4237e4639c108308fe13147b1c08af93c3d5724d9ff21ae797eb4b78fea3931", line 8218, in main 2024-02-20T12:56:42.567 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:56:42 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: r = ctx.func(ctx) 2024-02-20T12:56:42.567 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:56:42 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: File "/var/lib/ceph/680ee126-cfee-11ee-95bb-87774f69a715/cephadm.d4237e4639c108308fe13147b1c08af93c3d5724d9ff21ae797eb4b78fea3931", line 1759, in _default_image 2024-02-20T12:56:42.567 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:56:42 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: return func(ctx) 2024-02-20T12:56:42.567 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:56:42 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: File "/var/lib/ceph/680ee126-cfee-11ee-95bb-87774f69a715/cephadm.d4237e4639c108308fe13147b1c08af93c3d5724d9ff21ae797eb4b78fea3931", line 4344, in command_deploy 2024-02-20T12:56:42.568 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:56:42 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: uid, gid = extract_uid_gid_monitoring(ctx, daemon_type) 2024-02-20T12:56:42.568 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:56:42 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: File "/var/lib/ceph/680ee126-cfee-11ee-95bb-87774f69a715/cephadm.d4237e4639c108308fe13147b1c08af93c3d5724d9ff21ae797eb4b78fea3931", line 4275, in extract_uid_gid_monitoring 2024-02-20T12:56:42.568 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:56:42 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: uid, gid = extract_uid_gid(ctx, file_path=['/etc/alertmanager', '/etc/prometheus']) 2024-02-20T12:56:42.568 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:56:42 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: File "/var/lib/ceph/680ee126-cfee-11ee-95bb-87774f69a715/cephadm.d4237e4639c108308fe13147b1c08af93c3d5724d9ff21ae797eb4b78fea3931", line 2548, in extract_uid_gid 2024-02-20T12:56:42.568 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:56:42 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: raise RuntimeError('uid/gid not found') 2024-02-20T12:56:42.568 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:56:42 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: RuntimeError: uid/gid not found 2024-02-20T12:56:42.568 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:56:42 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cephadm 2024-02-20T12:56:40.916138+0000 mgr.smithi064.vriyor (mgr.14156) 177 : cephadm [ERR] Failed while placing alertmanager.smithi064 on smithi064: cephadm exited with an error code: 1, stderr:Non-zero exit code 125 from /bin/podman container inspect --format {{.State.Status}} ceph-680ee126-cfee-11ee-95bb-87774f69a715-alertmanager.smithi064 2024-02-20T12:56:42.568 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:56:42 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: /bin/podman: stderr Error: no such container ceph-680ee126-cfee-11ee-95bb-87774f69a715-alertmanager.smithi064 2024-02-20T12:56:42.568 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:56:42 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: Deploy daemon alertmanager.smithi064 ... 2024-02-20T12:56:42.568 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:56:42 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: Non-zero exit code 125 from /bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint stat --init -e CONTAINER_IMAGE=docker.io/prom/alertmanager:v0.20.0 -e NODE_NAME=smithi064 -e CEPH_USE_RANDOM_NONCE=1 docker.io/prom/alertmanager:v0.20.0 -c %u %g /etc/alertmanager 2024-02-20T12:56:42.568 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:56:42 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: stat: stderr Trying to pull docker.io/prom/alertmanager:v0.20.0... 2024-02-20T12:56:42.568 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:56:42 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: stat: stderr time="2024-02-20T12:52:19Z" level=warning msg="Failed, retrying in 1s ... (1/3). Error: copying system image from manifest list: determining manifest MIME type for docker://prom/alertmanager:v0.20.0: reading manifest sha256:b9323917a2eda265bec69e59a457f001c529facbbc8166df277f4850cdac61a0 in docker.io/prom/alertmanager: toomanyrequests: You have reached your pull rate limit. You may increase the limit by authenticating and upgrading: https://www.docker.com/increase-rate-limit" 2024-02-20T12:56:42.568 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:56:42 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: stat: stderr time="2024-02-20T12:52:56Z" level=warning msg="Failed, retrying in 1s ... (2/3). Error: initializing source docker://prom/alertmanager:v0.20.0: (Mirrors also failed: [docker-mirror.front.sepia.ceph.com:5000/prom/alertmanager:v0.20.0: pinging container registry docker-mirror.front.sepia.ceph.com:5000: Get \"https://docker-mirror.front.sepia.ceph.com:5000/v2/\": dial tcp 172.21.0.79:5000: connect: no route to host]): docker.io/prom/alertmanager:v0.20.0: reading manifest v0.20.0 in docker.io/prom/alertmanager: toomanyrequests: You have reached your pull rate limit. You may increase the limit by authenticating and upgrading: https://www.docker.com/increase-rate-limit" 2024-02-20T12:56:42.569 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:56:42 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: stat: stderr time="2024-02-20T12:53:34Z" level=warning msg="Failed, retrying in 1s ... (3/3). Error: initializing source docker://prom/alertmanager:v0.20.0: (Mirrors also failed: [docker-mirror.front.sepia.ceph.com:5000/prom/alertmanager:v0.20.0: pinging container registry docker-mirror.front.sepia.ceph.com:5000: Get \"https://docker-mirror.front.sepia.ceph.com:5000/v2/\": dial tcp 172.21.0.79:5000: connect: no route to host]): docker.io/prom/alertmanager:v0.20.0: reading manifest v0.20.0 in docker.io/prom/alertmanager: toomanyrequests: You have reached your pull rate limit. You may increase the limit by authenticating and upgrading: https://www.docker.com/increase-rate-limit" 2024-02-20T12:56:42.569 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:56:42 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: stat: stderr Error: copying system image from manifest list: determining manifest MIME type for docker://prom/alertmanager:v0.20.0: reading manifest sha256:b9323917a2eda265bec69e59a457f001c529facbbc8166df277f4850cdac61a0 in docker.io/prom/alertmanager: toomanyrequests: You have reached your pull rate limit. You may increase the limit by authenticating and upgrading: https://www.docker.com/increase-rate-limit 2024-02-20T12:56:42.569 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:56:42 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: Non-zero exit code 125 from /bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint stat --init -e CONTAINER_IMAGE=docker.io/prom/alertmanager:v0.20.0 -e NODE_NAME=smithi064 -e CEPH_USE_RANDOM_NONCE=1 docker.io/prom/alertmanager:v0.20.0 -c %u %g /etc/prometheus 2024-02-20T12:56:42.569 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:56:42 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: stat: stderr Trying to pull docker.io/prom/alertmanager:v0.20.0... 2024-02-20T12:56:42.569 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:56:42 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: stat: stderr time="2024-02-20T12:54:48Z" level=warning msg="Failed, retrying in 1s ... (1/3). Error: initializing source docker://prom/alertmanager:v0.20.0: (Mirrors also failed: [docker-mirror.front.sepia.ceph.com:5000/prom/alertmanager:v0.20.0: pinging container registry docker-mirror.front.sepia.ceph.com:5000: Get \"https://docker-mirror.front.sepia.ceph.com:5000/v2/\": dial tcp 172.21.0.79:5000: connect: no route to host]): docker.io/prom/alertmanager:v0.20.0: reading manifest v0.20.0 in docker.io/prom/alertmanager: toomanyrequests: You have reached your pull rate limit. You may increase the limit by authenticating and upgrading: https://www.docker.com/increase-rate-limit" 2024-02-20T12:56:42.569 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:56:42 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: stat: stderr time="2024-02-20T12:55:25Z" level=warning msg="Failed, retrying in 1s ... (2/3). Error: initializing source docker://prom/alertmanager:v0.20.0: (Mirrors also failed: [docker-mirror.front.sepia.ceph.com:5000/prom/alertmanager:v0.20.0: pinging container registry docker-mirror.front.sepia.ceph.com:5000: Get \"https://docker-mirror.front.sepia.ceph.com:5000/v2/\": dial tcp 172.21.0.79:5000: connect: no route to host]): docker.io/prom/alertmanager:v0.20.0: reading manifest v0.20.0 in docker.io/prom/alertmanager: toomanyrequests: You have reached your pull rate limit. You may increase the limit by authenticating and upgrading: https://www.docker.com/increase-rate-limit" 2024-02-20T12:56:42.569 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:56:42 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: stat: stderr time="2024-02-20T12:56:03Z" level=warning msg="Failed, retrying in 1s ... (3/3). Error: initializing source docker://prom/alertmanager:v0.20.0: (Mirrors also failed: [docker-mirror.front.sepia.ceph.com:5000/prom/alertmanager:v0.20.0: pinging container registry docker-mirror.front.sepia.ceph.com:5000: Get \"https://docker-mirror.front.sepia.ceph.com:5000/v2/\": dial tcp 172.21.0.79:5000: connect: no route to host]): docker.io/prom/alertmanager:v0.20.0: reading manifest v0.20.0 in docker.io/prom/alertmanager: toomanyrequests: You have reached your pull rate limit. You may increase the limit by authenticating and upgrading: https://www.docker.com/increase-rate-limit" 2024-02-20T12:56:42.569 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:56:42 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: stat: stderr Error: initializing source docker://prom/alertmanager:v0.20.0: (Mirrors also failed: [docker-mirror.front.sepia.ceph.com:5000/prom/alertmanager:v0.20.0: pinging container registry docker-mirror.front.sepia.ceph.com:5000: Get "https://docker-mirror.front.sepia.ceph.com:5000/v2/": dial tcp 172.21.0.79:5000: connect: no route to host]): docker.io/prom/alertmanager:v0.20.0: reading manifest v0.20.0 in docker.io/prom/alertmanager: toomanyrequests: You have reached your pull rate limit. You may increase the limit by authenticating and upgrading: https://www.docker.com/increase-rate-limit 2024-02-20T12:56:42.569 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:56:42 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: Traceback (most recent call last): 2024-02-20T12:56:42.569 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:56:42 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: File "/var/lib/ceph/680ee126-cfee-11ee-95bb-87774f69a715/cephadm.d4237e4639c108308fe13147b1c08af93c3d5724d9ff21ae797eb4b78fea3931", line 8230, in 2024-02-20T12:56:42.569 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:56:42 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: main() 2024-02-20T12:56:42.569 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:56:42 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: File "/var/lib/ceph/680ee126-cfee-11ee-95bb-87774f69a715/cephadm.d4237e4639c108308fe13147b1c08af93c3d5724d9ff21ae797eb4b78fea3931", line 8218, in main 2024-02-20T12:56:42.569 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:56:42 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: r = ctx.func(ctx) 2024-02-20T12:56:42.569 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:56:42 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: File "/var/lib/ceph/680ee126-cfee-11ee-95bb-87774f69a715/cephadm.d4237e4639c108308fe13147b1c08af93c3d5724d9ff21ae797eb4b78fea3931", line 1759, in _default_image 2024-02-20T12:56:42.570 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:56:42 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: return func(ctx) 2024-02-20T12:56:42.570 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:56:42 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: File "/var/lib/ceph/680ee126-cfee-11ee-95bb-87774f69a715/cephadm.d4237e4639c108308fe13147b1c08af93c3d5724d9ff21ae797eb4b78fea3931", line 4344, in command_deploy 2024-02-20T12:56:42.570 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:56:42 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: uid, gid = extract_uid_gid_monitoring(ctx, daemon_type) 2024-02-20T12:56:42.570 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:56:42 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: File "/var/lib/ceph/680ee126-cfee-11ee-95bb-87774f69a715/cephadm.d4237e4639c108308fe13147b1c08af93c3d5724d9ff21ae797eb4b78fea3931", line 4275, in extract_uid_gid_monitoring 2024-02-20T12:56:42.570 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:56:42 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: uid, gid = extract_uid_gid(ctx, file_path=['/etc/alertmanager', '/etc/prometheus']) 2024-02-20T12:56:42.570 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:56:42 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: File "/var/lib/ceph/680ee126-cfee-11ee-95bb-87774f69a715/cephadm.d4237e4639c108308fe13147b1c08af93c3d5724d9ff21ae797eb4b78fea3931", line 2548, in extract_uid_gid 2024-02-20T12:56:42.570 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:56:42 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: raise RuntimeError('uid/gid not found') 2024-02-20T12:56:42.570 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:56:42 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: RuntimeError: uid/gid not found 2024-02-20T12:56:42.570 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:56:42 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cephadm 2024-02-20T12:56:40.922424+0000 mgr.smithi064.vriyor (mgr.14156) 178 : cephadm [INF] Deploying daemon crash.smithi064 on smithi064 2024-02-20T12:56:43.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:56:43 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:56:41.757646+0000 mgr.smithi064.vriyor (mgr.14156) 179 : cluster [DBG] pgmap v168: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:56:43.441 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:43.439+0000 7f685af57700 1 -- 172.21.15.71:0/1870940888 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f68540f6590 msgr2=0x7f68540f69b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:56:43.441 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:43.439+0000 7f685af57700 1 --2- 172.21.15.71:0/1870940888 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f68540f6590 0x7f68540f69b0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 rx=0x7f6848004760 tx=0x7f6848009ad0).stop 2024-02-20T12:56:43.441 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:43.439+0000 7f685af57700 1 -- 172.21.15.71:0/1870940888 shutdown_connections 2024-02-20T12:56:43.441 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:43.439+0000 7f685af57700 1 --2- 172.21.15.71:0/1870940888 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f68540f6590 0x7f68540f69b0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:56:43.441 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:43.439+0000 7f685af57700 1 -- 172.21.15.71:0/1870940888 >> 172.21.15.71:0/1870940888 conn(0x7f68540f1cd0 msgr2=0x7f68540f4110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:56:43.441 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:43.439+0000 7f685af57700 1 -- 172.21.15.71:0/1870940888 shutdown_connections 2024-02-20T12:56:43.441 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:43.439+0000 7f685af57700 1 -- 172.21.15.71:0/1870940888 wait complete. 2024-02-20T12:56:43.443 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:43.440+0000 7f685af57700 1 Processor -- start 2024-02-20T12:56:43.443 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:43.440+0000 7f685af57700 1 -- start start 2024-02-20T12:56:43.443 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:43.440+0000 7f685af57700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6854106930 0x7f6854106d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:56:43.443 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:43.440+0000 7f685af57700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6854107240 con 0x7f6854106930 2024-02-20T12:56:43.443 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:43.440+0000 7f6858cf3700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6854106930 0x7f6854106d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:56:43.443 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:43.440+0000 7f6858cf3700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6854106930 0x7f6854106d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.71:38128/0 (socket says 172.21.15.71:38128) 2024-02-20T12:56:43.443 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:43.440+0000 7f6858cf3700 1 -- 172.21.15.71:0/324316723 learned_addr learned my addr 172.21.15.71:0/324316723 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-02-20T12:56:43.443 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:43.441+0000 7f6858cf3700 1 -- 172.21.15.71:0/324316723 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6848009d70 con 0x7f6854106930 2024-02-20T12:56:43.443 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:43.441+0000 7f6858cf3700 1 --2- 172.21.15.71:0/324316723 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6854106930 0x7f6854106d50 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 rx=0x7f68540f7690 tx=0x7f6848009ad0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:56:43.443 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:43.441+0000 7f68517fa700 1 -- 172.21.15.71:0/324316723 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f684800e040 con 0x7f6854106930 2024-02-20T12:56:43.444 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:43.441+0000 7f68517fa700 1 -- 172.21.15.71:0/324316723 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(25 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f68480095b0 con 0x7f6854106930 2024-02-20T12:56:43.445 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:43.441+0000 7f685af57700 1 -- 172.21.15.71:0/324316723 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6854107440 con 0x7f6854106930 2024-02-20T12:56:43.445 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:43.441+0000 7f68517fa700 1 -- 172.21.15.71:0/324316723 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f684801fa80 con 0x7f6854106930 2024-02-20T12:56:43.445 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:43.441+0000 7f685af57700 1 -- 172.21.15.71:0/324316723 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6854107f40 con 0x7f6854106930 2024-02-20T12:56:43.445 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:43.442+0000 7f68517fa700 1 -- 172.21.15.71:0/324316723 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 37151+0+0 (secure 0 0 0) 0x7f6848026060 con 0x7f6854106930 2024-02-20T12:56:43.445 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:43.442+0000 7f68517fa700 1 --2- 172.21.15.71:0/324316723 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f6844031090 0x7f6844033550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:56:43.445 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:43.442+0000 7f68517fa700 1 -- 172.21.15.71:0/324316723 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1065+0+0 (secure 0 0 0) 0x7f6848042130 con 0x7f6854106930 2024-02-20T12:56:43.445 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:43.442+0000 7f685af57700 1 -- 172.21.15.71:0/324316723 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68540591e0 con 0x7f6854106930 2024-02-20T12:56:43.445 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:43.442+0000 7f6853fff700 1 --2- 172.21.15.71:0/324316723 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f6844031090 0x7f6844033550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:56:43.445 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:43.442+0000 7f6853fff700 1 --2- 172.21.15.71:0/324316723 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f6844031090 0x7f6844033550 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 rx=0x7f68400096a0 tx=0x7f6840006b40).ready entity=mgr.14156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:56:43.448 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:43.447+0000 7f68517fa700 1 -- 172.21.15.71:0/324316723 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+153237 (secure 0 0 0) 0x7f684801e560 con 0x7f6854106930 2024-02-20T12:56:43.656 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:43.655+0000 7f685af57700 1 -- 172.21.15.71:0/324316723 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f68541085a0 con 0x7f6854106930 2024-02-20T12:56:43.657 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:43.656+0000 7f68517fa700 1 -- 172.21.15.71:0/324316723 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7f684801e380 con 0x7f6854106930 2024-02-20T12:56:43.657 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-20T12:56:43.657 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":1,"fsid":"680ee126-cfee-11ee-95bb-87774f69a715","modified":"2024-02-20T12:49:44.699059Z","created":"2024-02-20T12:49:44.699059Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi064","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:3300","nonce":0},{"type":"v1","addr":"172.21.15.64:6789","nonce":0}]},"addr":"172.21.15.64:6789/0","public_addr":"172.21.15.64:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-20T12:56:43.659 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:43.657+0000 7f685af57700 1 -- 172.21.15.71:0/324316723 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f6844031090 msgr2=0x7f6844033550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:56:43.659 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:43.657+0000 7f685af57700 1 --2- 172.21.15.71:0/324316723 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f6844031090 0x7f6844033550 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 rx=0x7f68400096a0 tx=0x7f6840006b40).stop 2024-02-20T12:56:43.659 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:43.657+0000 7f685af57700 1 -- 172.21.15.71:0/324316723 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6854106930 msgr2=0x7f6854106d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:56:43.659 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:43.657+0000 7f685af57700 1 --2- 172.21.15.71:0/324316723 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6854106930 0x7f6854106d50 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 rx=0x7f68540f7690 tx=0x7f6848009ad0).stop 2024-02-20T12:56:43.659 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:43.657+0000 7f685af57700 1 -- 172.21.15.71:0/324316723 shutdown_connections 2024-02-20T12:56:43.659 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:43.657+0000 7f685af57700 1 --2- 172.21.15.71:0/324316723 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f6844031090 0x7f6844033550 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:56:43.659 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:43.657+0000 7f685af57700 1 --2- 172.21.15.71:0/324316723 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6854106930 0x7f6854106d50 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:56:43.659 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:43.657+0000 7f685af57700 1 -- 172.21.15.71:0/324316723 >> 172.21.15.71:0/324316723 conn(0x7f68540f1cd0 msgr2=0x7f68540f28a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:56:43.659 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:43.657+0000 7f685af57700 1 -- 172.21.15.71:0/324316723 shutdown_connections 2024-02-20T12:56:43.659 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:43.657+0000 7f685af57700 1 -- 172.21.15.71:0/324316723 wait complete. 2024-02-20T12:56:43.660 INFO:teuthology.orchestra.run.smithi071.stderr:dumped monmap epoch 1 2024-02-20T12:56:44.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:56:44 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:56:43.657446+0000 mon.smithi064 (mon.0) 203 : audit [DBG] from='client.? 172.21.15.71:0/324316723' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-20T12:56:44.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:56:44 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:56:43.676982+0000 mon.smithi064 (mon.0) 204 : audit [INF] from='mgr.14156 172.21.15.64:0/3284974785' entity='mgr.smithi064.vriyor' 2024-02-20T12:56:44.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:56:44 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:56:43.787980+0000 mon.smithi064 (mon.0) 205 : audit [INF] from='mgr.14156 172.21.15.64:0/3284974785' entity='mgr.smithi064.vriyor' 2024-02-20T12:56:44.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:56:44 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:56:43.793444+0000 mon.smithi064 (mon.0) 206 : audit [INF] from='mgr.14156 172.21.15.64:0/3284974785' entity='mgr.smithi064.vriyor' 2024-02-20T12:56:44.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:56:44 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:56:43.793853+0000 mon.smithi064 (mon.0) 207 : audit [INF] from='mgr.14156 172.21.15.64:0/3284974785' entity='mgr.smithi064.vriyor' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-02-20T12:56:44.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:56:44 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:56:43.799633+0000 mon.smithi064 (mon.0) 208 : audit [INF] from='mgr.14156 172.21.15.64:0/3284974785' entity='mgr.smithi064.vriyor' 2024-02-20T12:56:44.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:56:44 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:56:43.848244+0000 mon.smithi064 (mon.0) 209 : audit [INF] from='mgr.14156 172.21.15.64:0/3284974785' entity='mgr.smithi064.vriyor' 2024-02-20T12:56:45.482 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-20T12:56:45.483 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/daemon-base:latest-pacific shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 680ee126-cfee-11ee-95bb-87774f69a715 -- ceph mon dump -f json 2024-02-20T12:56:45.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:56:45 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:56:43.757841+0000 mgr.smithi064.vriyor (mgr.14156) 180 : cluster [DBG] pgmap v169: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:56:45.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:56:45 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:56:43.794183+0000 mgr.smithi064.vriyor (mgr.14156) 181 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-02-20T12:56:45.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:56:45 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cephadm 2024-02-20T12:56:43.802607+0000 mgr.smithi064.vriyor (mgr.14156) 182 : cephadm [INF] Deploying daemon grafana.smithi064 on smithi064 2024-02-20T12:56:47.388 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:47.385+0000 7ffa0146d700 1 -- 172.21.15.71:0/904145768 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff9fc0f5480 msgr2=0x7ff9fc0f58a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:56:47.388 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:47.385+0000 7ffa0146d700 1 --2- 172.21.15.71:0/904145768 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff9fc0f5480 0x7ff9fc0f58a0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 rx=0x7ff9f0009fb0 tx=0x7ff9f0009b30).stop 2024-02-20T12:56:47.388 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:47.385+0000 7ffa0146d700 1 -- 172.21.15.71:0/904145768 shutdown_connections 2024-02-20T12:56:47.388 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:47.385+0000 7ffa0146d700 1 --2- 172.21.15.71:0/904145768 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff9fc0f5480 0x7ff9fc0f58a0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:56:47.388 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:47.385+0000 7ffa0146d700 1 -- 172.21.15.71:0/904145768 >> 172.21.15.71:0/904145768 conn(0x7ff9fc0f0be0 msgr2=0x7ff9fc0f3000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:56:47.388 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:47.385+0000 7ffa0146d700 1 -- 172.21.15.71:0/904145768 shutdown_connections 2024-02-20T12:56:47.388 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:47.385+0000 7ffa0146d700 1 -- 172.21.15.71:0/904145768 wait complete. 2024-02-20T12:56:47.388 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:47.386+0000 7ffa0146d700 1 Processor -- start 2024-02-20T12:56:47.388 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:47.386+0000 7ffa0146d700 1 -- start start 2024-02-20T12:56:47.388 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:47.386+0000 7ffa0146d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff9fc0f5480 0x7ff9fc0fd460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:56:47.388 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:47.386+0000 7ffa0146d700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9fc0fb8e0 con 0x7ff9fc0f5480 2024-02-20T12:56:47.389 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:47.386+0000 7ff9faffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff9fc0f5480 0x7ff9fc0fd460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:56:47.389 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:47.386+0000 7ff9faffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff9fc0f5480 0x7ff9fc0fd460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.71:44874/0 (socket says 172.21.15.71:44874) 2024-02-20T12:56:47.389 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:47.386+0000 7ff9faffd700 1 -- 172.21.15.71:0/3929261836 learned_addr learned my addr 172.21.15.71:0/3929261836 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-02-20T12:56:47.389 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:47.386+0000 7ff9faffd700 1 -- 172.21.15.71:0/3929261836 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9f0005040 con 0x7ff9fc0f5480 2024-02-20T12:56:47.389 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:47.387+0000 7ff9faffd700 1 --2- 172.21.15.71:0/3929261836 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff9fc0f5480 0x7ff9fc0fd460 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 rx=0x7ff9f00089f0 tx=0x7ff9f000b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:56:47.390 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:47.387+0000 7ff9e37fe700 1 -- 172.21.15.71:0/3929261836 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff9f0009320 con 0x7ff9fc0f5480 2024-02-20T12:56:47.390 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:47.387+0000 7ffa0146d700 1 -- 172.21.15.71:0/3929261836 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9fc0fba80 con 0x7ff9fc0f5480 2024-02-20T12:56:47.390 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:47.387+0000 7ff9e37fe700 1 -- 172.21.15.71:0/3929261836 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(25 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7ff9f0013ec0 con 0x7ff9fc0f5480 2024-02-20T12:56:47.390 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:47.387+0000 7ff9e37fe700 1 -- 172.21.15.71:0/3929261836 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff9f0014d30 con 0x7ff9fc0f5480 2024-02-20T12:56:47.390 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:47.387+0000 7ffa0146d700 1 -- 172.21.15.71:0/3929261836 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9fc0fbf10 con 0x7ff9fc0f5480 2024-02-20T12:56:47.390 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:47.388+0000 7ff9e37fe700 1 -- 172.21.15.71:0/3929261836 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 37151+0+0 (secure 0 0 0) 0x7ff9f0014420 con 0x7ff9fc0f5480 2024-02-20T12:56:47.391 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:47.388+0000 7ff9e37fe700 1 --2- 172.21.15.71:0/3929261836 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7ff9e4031090 0x7ff9e4033550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:56:47.391 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:47.388+0000 7ffa0146d700 1 -- 172.21.15.71:0/3929261836 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9fc0591e0 con 0x7ff9fc0f5480 2024-02-20T12:56:47.391 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:47.388+0000 7ff9e37fe700 1 -- 172.21.15.71:0/3929261836 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1065+0+0 (secure 0 0 0) 0x7ff9f00424a0 con 0x7ff9fc0f5480 2024-02-20T12:56:47.391 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:47.388+0000 7ff9fa7fc700 1 --2- 172.21.15.71:0/3929261836 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7ff9e4031090 0x7ff9e4033550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:56:47.391 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:47.388+0000 7ff9fa7fc700 1 --2- 172.21.15.71:0/3929261836 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7ff9e4031090 0x7ff9e4033550 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 rx=0x7ff9ec0096a0 tx=0x7ff9ec006b40).ready entity=mgr.14156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:56:47.394 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:47.393+0000 7ff9e37fe700 1 -- 172.21.15.71:0/3929261836 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+153237 (secure 0 0 0) 0x7ff9f0014750 con 0x7ff9fc0f5480 2024-02-20T12:56:47.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:56:47 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:56:45.758182+0000 mgr.smithi064.vriyor (mgr.14156) 183 : cluster [DBG] pgmap v170: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:56:47.599 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:47.598+0000 7ffa0146d700 1 -- 172.21.15.71:0/3929261836 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7ff9fc0faf70 con 0x7ff9fc0f5480 2024-02-20T12:56:47.600 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:47.598+0000 7ff9e37fe700 1 -- 172.21.15.71:0/3929261836 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7ff9f001d7f0 con 0x7ff9fc0f5480 2024-02-20T12:56:47.600 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-20T12:56:47.600 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":1,"fsid":"680ee126-cfee-11ee-95bb-87774f69a715","modified":"2024-02-20T12:49:44.699059Z","created":"2024-02-20T12:49:44.699059Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi064","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:3300","nonce":0},{"type":"v1","addr":"172.21.15.64:6789","nonce":0}]},"addr":"172.21.15.64:6789/0","public_addr":"172.21.15.64:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-20T12:56:47.601 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:47.600+0000 7ffa0146d700 1 -- 172.21.15.71:0/3929261836 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7ff9e4031090 msgr2=0x7ff9e4033550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:56:47.602 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:47.600+0000 7ffa0146d700 1 --2- 172.21.15.71:0/3929261836 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7ff9e4031090 0x7ff9e4033550 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 rx=0x7ff9ec0096a0 tx=0x7ff9ec006b40).stop 2024-02-20T12:56:47.602 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:47.600+0000 7ffa0146d700 1 -- 172.21.15.71:0/3929261836 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff9fc0f5480 msgr2=0x7ff9fc0fd460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:56:47.602 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:47.600+0000 7ffa0146d700 1 --2- 172.21.15.71:0/3929261836 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff9fc0f5480 0x7ff9fc0fd460 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 rx=0x7ff9f00089f0 tx=0x7ff9f000b040).stop 2024-02-20T12:56:47.602 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:47.600+0000 7ffa0146d700 1 -- 172.21.15.71:0/3929261836 shutdown_connections 2024-02-20T12:56:47.602 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:47.600+0000 7ffa0146d700 1 --2- 172.21.15.71:0/3929261836 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7ff9e4031090 0x7ff9e4033550 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:56:47.602 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:47.600+0000 7ffa0146d700 1 --2- 172.21.15.71:0/3929261836 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff9fc0f5480 0x7ff9fc0fd460 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:56:47.602 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:47.600+0000 7ffa0146d700 1 -- 172.21.15.71:0/3929261836 >> 172.21.15.71:0/3929261836 conn(0x7ff9fc0f0be0 msgr2=0x7ff9fc05fee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:56:47.602 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:47.600+0000 7ffa0146d700 1 -- 172.21.15.71:0/3929261836 shutdown_connections 2024-02-20T12:56:47.602 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:47.600+0000 7ffa0146d700 1 -- 172.21.15.71:0/3929261836 wait complete. 2024-02-20T12:56:47.603 INFO:teuthology.orchestra.run.smithi071.stderr:dumped monmap epoch 1 2024-02-20T12:56:48.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:56:48 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:56:47.600150+0000 mon.smithi064 (mon.0) 210 : audit [DBG] from='client.? 172.21.15.71:0/3929261836' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-20T12:56:49.186 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-20T12:56:49.186 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/daemon-base:latest-pacific shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 680ee126-cfee-11ee-95bb-87774f69a715 -- ceph mon dump -f json 2024-02-20T12:56:49.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:56:49 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:56:47.758437+0000 mgr.smithi064.vriyor (mgr.14156) 184 : cluster [DBG] pgmap v171: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:56:51.233 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:51.205+0000 7fedc0183700 1 -- 172.21.15.71:0/3688201382 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fedb80f5480 msgr2=0x7fedb80f58a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:56:51.234 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:51.205+0000 7fedc0183700 1 --2- 172.21.15.71:0/3688201382 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fedb80f5480 0x7fedb80f58a0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 rx=0x7feda8009fb0 tx=0x7feda8009b30).stop 2024-02-20T12:56:51.234 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:51.205+0000 7fedc0183700 1 -- 172.21.15.71:0/3688201382 shutdown_connections 2024-02-20T12:56:51.234 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:51.205+0000 7fedc0183700 1 --2- 172.21.15.71:0/3688201382 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fedb80f5480 0x7fedb80f58a0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:56:51.234 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:51.205+0000 7fedc0183700 1 -- 172.21.15.71:0/3688201382 >> 172.21.15.71:0/3688201382 conn(0x7fedb80f0be0 msgr2=0x7fedb80f3000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:56:51.234 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:51.205+0000 7fedc0183700 1 -- 172.21.15.71:0/3688201382 shutdown_connections 2024-02-20T12:56:51.234 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:51.205+0000 7fedc0183700 1 -- 172.21.15.71:0/3688201382 wait complete. 2024-02-20T12:56:51.235 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:51.206+0000 7fedc0183700 1 Processor -- start 2024-02-20T12:56:51.235 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:51.206+0000 7fedc0183700 1 -- start start 2024-02-20T12:56:51.235 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:51.206+0000 7fedc0183700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fedb80f5480 0x7fedb81094f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:56:51.235 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:51.206+0000 7fedc0183700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedb8060320 con 0x7fedb80f5480 2024-02-20T12:56:51.235 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:51.206+0000 7fedbdf1f700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fedb80f5480 0x7fedb81094f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:56:51.235 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:51.206+0000 7fedbdf1f700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fedb80f5480 0x7fedb81094f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.71:44880/0 (socket says 172.21.15.71:44880) 2024-02-20T12:56:51.236 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:51.206+0000 7fedbdf1f700 1 -- 172.21.15.71:0/546788602 learned_addr learned my addr 172.21.15.71:0/546788602 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-02-20T12:56:51.236 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:51.206+0000 7fedbdf1f700 1 -- 172.21.15.71:0/546788602 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feda8005040 con 0x7fedb80f5480 2024-02-20T12:56:51.236 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:51.206+0000 7fedbdf1f700 1 --2- 172.21.15.71:0/546788602 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fedb80f5480 0x7fedb81094f0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 rx=0x7feda80089f0 tx=0x7feda800b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:56:51.236 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:51.206+0000 7fedae7fc700 1 -- 172.21.15.71:0/546788602 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7feda8009320 con 0x7fedb80f5480 2024-02-20T12:56:51.236 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:51.206+0000 7fedae7fc700 1 -- 172.21.15.71:0/546788602 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(25 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7feda8013ec0 con 0x7fedb80f5480 2024-02-20T12:56:51.236 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:51.206+0000 7fedae7fc700 1 -- 172.21.15.71:0/546788602 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7feda8014d30 con 0x7fedb80f5480 2024-02-20T12:56:51.237 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:51.206+0000 7fedc0183700 1 -- 172.21.15.71:0/546788602 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fedb8060520 con 0x7fedb80f5480 2024-02-20T12:56:51.237 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:51.206+0000 7fedc0183700 1 -- 172.21.15.71:0/546788602 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fedb8109ba0 con 0x7fedb80f5480 2024-02-20T12:56:51.237 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:51.207+0000 7fedae7fc700 1 -- 172.21.15.71:0/546788602 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 37151+0+0 (secure 0 0 0) 0x7feda80263a0 con 0x7fedb80f5480 2024-02-20T12:56:51.237 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:51.207+0000 7fedae7fc700 1 --2- 172.21.15.71:0/546788602 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7feda40310e0 0x7feda40335a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:56:51.237 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:51.207+0000 7fedae7fc700 1 -- 172.21.15.71:0/546788602 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1065+0+0 (secure 0 0 0) 0x7feda8042420 con 0x7fedb80f5480 2024-02-20T12:56:51.237 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:51.207+0000 7fedbd71e700 1 --2- 172.21.15.71:0/546788602 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7feda40310e0 0x7feda40335a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:56:51.237 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:51.207+0000 7fedc0183700 1 -- 172.21.15.71:0/546788602 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fedb80591e0 con 0x7fedb80f5480 2024-02-20T12:56:51.238 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:51.208+0000 7fedbd71e700 1 --2- 172.21.15.71:0/546788602 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7feda40310e0 0x7feda40335a0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 rx=0x7fedb40096a0 tx=0x7fedb4006b40).ready entity=mgr.14156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:56:51.238 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:51.212+0000 7fedae7fc700 1 -- 172.21.15.71:0/546788602 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+153237 (secure 0 0 0) 0x7feda80266d0 con 0x7fedb80f5480 2024-02-20T12:56:51.421 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:51.420+0000 7fedc0183700 1 -- 172.21.15.71:0/546788602 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fedb80faf70 con 0x7fedb80f5480 2024-02-20T12:56:51.422 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:51.420+0000 7fedae7fc700 1 -- 172.21.15.71:0/546788602 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7feda802b020 con 0x7fedb80f5480 2024-02-20T12:56:51.422 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-20T12:56:51.422 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":1,"fsid":"680ee126-cfee-11ee-95bb-87774f69a715","modified":"2024-02-20T12:49:44.699059Z","created":"2024-02-20T12:49:44.699059Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi064","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:3300","nonce":0},{"type":"v1","addr":"172.21.15.64:6789","nonce":0}]},"addr":"172.21.15.64:6789/0","public_addr":"172.21.15.64:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-20T12:56:51.423 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:51.422+0000 7fedc0183700 1 -- 172.21.15.71:0/546788602 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7feda40310e0 msgr2=0x7feda40335a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:56:51.424 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:51.422+0000 7fedc0183700 1 --2- 172.21.15.71:0/546788602 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7feda40310e0 0x7feda40335a0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 rx=0x7fedb40096a0 tx=0x7fedb4006b40).stop 2024-02-20T12:56:51.424 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:51.422+0000 7fedc0183700 1 -- 172.21.15.71:0/546788602 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fedb80f5480 msgr2=0x7fedb81094f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:56:51.424 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:51.422+0000 7fedc0183700 1 --2- 172.21.15.71:0/546788602 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fedb80f5480 0x7fedb81094f0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 rx=0x7feda80089f0 tx=0x7feda800b040).stop 2024-02-20T12:56:51.424 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:51.422+0000 7fedc0183700 1 -- 172.21.15.71:0/546788602 shutdown_connections 2024-02-20T12:56:51.424 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:51.422+0000 7fedc0183700 1 --2- 172.21.15.71:0/546788602 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7feda40310e0 0x7feda40335a0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:56:51.424 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:51.422+0000 7fedc0183700 1 --2- 172.21.15.71:0/546788602 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fedb80f5480 0x7fedb81094f0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:56:51.424 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:51.422+0000 7fedc0183700 1 -- 172.21.15.71:0/546788602 >> 172.21.15.71:0/546788602 conn(0x7fedb80f0be0 msgr2=0x7fedb80f16d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:56:51.425 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:51.422+0000 7fedc0183700 1 -- 172.21.15.71:0/546788602 shutdown_connections 2024-02-20T12:56:51.425 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:51.422+0000 7fedc0183700 1 -- 172.21.15.71:0/546788602 wait complete. 2024-02-20T12:56:51.425 INFO:teuthology.orchestra.run.smithi071.stderr:dumped monmap epoch 1 2024-02-20T12:56:51.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:56:51 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:56:49.758705+0000 mgr.smithi064.vriyor (mgr.14156) 185 : cluster [DBG] pgmap v172: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:56:52.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:56:52 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:56:51.421896+0000 mon.smithi064 (mon.0) 211 : audit [DBG] from='client.? 172.21.15.71:0/546788602' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-20T12:56:53.157 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-20T12:56:53.158 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/daemon-base:latest-pacific shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 680ee126-cfee-11ee-95bb-87774f69a715 -- ceph mon dump -f json 2024-02-20T12:56:53.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:56:53 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:56:51.758894+0000 mgr.smithi064.vriyor (mgr.14156) 186 : cluster [DBG] pgmap v173: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:56:54.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:56:54 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:56:53.848108+0000 mon.smithi064 (mon.0) 212 : audit [INF] from='mgr.14156 172.21.15.64:0/3284974785' entity='mgr.smithi064.vriyor' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi064.vriyor/mirror_snapshot_schedule"}]: dispatch 2024-02-20T12:56:54.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:56:54 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:56:53.848746+0000 mon.smithi064 (mon.0) 213 : audit [INF] from='mgr.14156 172.21.15.64:0/3284974785' entity='mgr.smithi064.vriyor' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi064.vriyor/trash_purge_schedule"}]: dispatch 2024-02-20T12:56:55.213 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:55.209+0000 7fa3254ca700 1 -- 172.21.15.71:0/1428120029 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa3200f52d0 msgr2=0x7fa3200f56f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:56:55.213 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:55.209+0000 7fa3254ca700 1 --2- 172.21.15.71:0/1428120029 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa3200f52d0 0x7fa3200f56f0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 rx=0x7fa310009fb0 tx=0x7fa310009b30).stop 2024-02-20T12:56:55.213 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:55.209+0000 7fa3254ca700 1 -- 172.21.15.71:0/1428120029 shutdown_connections 2024-02-20T12:56:55.213 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:55.209+0000 7fa3254ca700 1 --2- 172.21.15.71:0/1428120029 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa3200f52d0 0x7fa3200f56f0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:56:55.214 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:55.209+0000 7fa3254ca700 1 -- 172.21.15.71:0/1428120029 >> 172.21.15.71:0/1428120029 conn(0x7fa3200f0a30 msgr2=0x7fa3200f2e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:56:55.214 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:55.209+0000 7fa3254ca700 1 -- 172.21.15.71:0/1428120029 shutdown_connections 2024-02-20T12:56:55.214 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:55.209+0000 7fa3254ca700 1 -- 172.21.15.71:0/1428120029 wait complete. 2024-02-20T12:56:55.214 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:55.210+0000 7fa3254ca700 1 Processor -- start 2024-02-20T12:56:55.214 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:55.210+0000 7fa3254ca700 1 -- start start 2024-02-20T12:56:55.214 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:55.210+0000 7fa3254ca700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa3200f52d0 0x7fa320106640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:56:55.214 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:55.210+0000 7fa3254ca700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa320106b30 con 0x7fa3200f52d0 2024-02-20T12:56:55.214 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:55.210+0000 7fa31ffff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa3200f52d0 0x7fa320106640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:56:55.214 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:55.210+0000 7fa31ffff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa3200f52d0 0x7fa320106640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.71:44888/0 (socket says 172.21.15.71:44888) 2024-02-20T12:56:55.214 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:55.210+0000 7fa31ffff700 1 -- 172.21.15.71:0/181624879 learned_addr learned my addr 172.21.15.71:0/181624879 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-02-20T12:56:55.214 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:55.210+0000 7fa31ffff700 1 -- 172.21.15.71:0/181624879 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa310005040 con 0x7fa3200f52d0 2024-02-20T12:56:55.215 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:55.211+0000 7fa31ffff700 1 --2- 172.21.15.71:0/181624879 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa3200f52d0 0x7fa320106640 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 rx=0x7fa3100089f0 tx=0x7fa31000b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:56:55.215 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:55.211+0000 7fa31cff9700 1 -- 172.21.15.71:0/181624879 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa310009320 con 0x7fa3200f52d0 2024-02-20T12:56:55.215 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:55.211+0000 7fa31cff9700 1 -- 172.21.15.71:0/181624879 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(25 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7fa310013ec0 con 0x7fa3200f52d0 2024-02-20T12:56:55.215 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:55.211+0000 7fa31cff9700 1 -- 172.21.15.71:0/181624879 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa310014d30 con 0x7fa3200f52d0 2024-02-20T12:56:55.215 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:55.211+0000 7fa3254ca700 1 -- 172.21.15.71:0/181624879 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa320106d30 con 0x7fa3200f52d0 2024-02-20T12:56:55.215 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:55.211+0000 7fa3254ca700 1 -- 172.21.15.71:0/181624879 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3201071c0 con 0x7fa3200f52d0 2024-02-20T12:56:55.215 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:55.211+0000 7fa31cff9700 1 -- 172.21.15.71:0/181624879 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 37151+0+0 (secure 0 0 0) 0x7fa3100263a0 con 0x7fa3200f52d0 2024-02-20T12:56:55.215 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:55.212+0000 7fa31cff9700 1 --2- 172.21.15.71:0/181624879 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fa3080310e0 0x7fa3080335a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:56:55.215 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:55.212+0000 7fa31cff9700 1 -- 172.21.15.71:0/181624879 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1065+0+0 (secure 0 0 0) 0x7fa310042420 con 0x7fa3200f52d0 2024-02-20T12:56:55.215 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:55.212+0000 7fa3254ca700 1 -- 172.21.15.71:0/181624879 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa320045bb0 con 0x7fa3200f52d0 2024-02-20T12:56:55.215 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:55.212+0000 7fa31f7fe700 1 --2- 172.21.15.71:0/181624879 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fa3080310e0 0x7fa3080335a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:56:55.215 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:55.212+0000 7fa31f7fe700 1 --2- 172.21.15.71:0/181624879 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fa3080310e0 0x7fa3080335a0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 rx=0x7fa3140096a0 tx=0x7fa314006b40).ready entity=mgr.14156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:56:55.217 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:55.217+0000 7fa31cff9700 1 -- 172.21.15.71:0/181624879 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+153237 (secure 0 0 0) 0x7fa3100266d0 con 0x7fa3200f52d0 2024-02-20T12:56:55.425 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:55.424+0000 7fa3254ca700 1 -- 172.21.15.71:0/181624879 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fa320107e40 con 0x7fa3200f52d0 2024-02-20T12:56:55.426 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:55.425+0000 7fa31cff9700 1 -- 172.21.15.71:0/181624879 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7fa310012330 con 0x7fa3200f52d0 2024-02-20T12:56:55.426 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-20T12:56:55.426 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":1,"fsid":"680ee126-cfee-11ee-95bb-87774f69a715","modified":"2024-02-20T12:49:44.699059Z","created":"2024-02-20T12:49:44.699059Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi064","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:3300","nonce":0},{"type":"v1","addr":"172.21.15.64:6789","nonce":0}]},"addr":"172.21.15.64:6789/0","public_addr":"172.21.15.64:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-20T12:56:55.427 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:55.427+0000 7fa3254ca700 1 -- 172.21.15.71:0/181624879 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fa3080310e0 msgr2=0x7fa3080335a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:56:55.427 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:55.427+0000 7fa3254ca700 1 --2- 172.21.15.71:0/181624879 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fa3080310e0 0x7fa3080335a0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 rx=0x7fa3140096a0 tx=0x7fa314006b40).stop 2024-02-20T12:56:55.428 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:55.427+0000 7fa3254ca700 1 -- 172.21.15.71:0/181624879 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa3200f52d0 msgr2=0x7fa320106640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:56:55.428 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:55.427+0000 7fa3254ca700 1 --2- 172.21.15.71:0/181624879 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa3200f52d0 0x7fa320106640 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 rx=0x7fa3100089f0 tx=0x7fa31000b040).stop 2024-02-20T12:56:55.428 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:55.427+0000 7fa3254ca700 1 -- 172.21.15.71:0/181624879 shutdown_connections 2024-02-20T12:56:55.428 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:55.427+0000 7fa3254ca700 1 --2- 172.21.15.71:0/181624879 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fa3080310e0 0x7fa3080335a0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:56:55.428 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:55.427+0000 7fa3254ca700 1 --2- 172.21.15.71:0/181624879 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa3200f52d0 0x7fa320106640 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:56:55.428 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:55.427+0000 7fa3254ca700 1 -- 172.21.15.71:0/181624879 >> 172.21.15.71:0/181624879 conn(0x7fa3200f0a30 msgr2=0x7fa3200f1550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:56:55.428 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:55.427+0000 7fa3254ca700 1 -- 172.21.15.71:0/181624879 shutdown_connections 2024-02-20T12:56:55.428 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:55.427+0000 7fa3254ca700 1 -- 172.21.15.71:0/181624879 wait complete. 2024-02-20T12:56:55.428 INFO:teuthology.orchestra.run.smithi071.stderr:dumped monmap epoch 1 2024-02-20T12:56:55.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:56:55 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:56:53.759094+0000 mgr.smithi064.vriyor (mgr.14156) 187 : cluster [DBG] pgmap v174: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:56:56.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:56:56 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:56:55.425850+0000 mon.smithi064 (mon.0) 214 : audit [DBG] from='client.? 172.21.15.71:0/181624879' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-20T12:56:57.205 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-20T12:56:57.205 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/daemon-base:latest-pacific shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 680ee126-cfee-11ee-95bb-87774f69a715 -- ceph mon dump -f json 2024-02-20T12:56:57.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:56:57 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:56:55.759353+0000 mgr.smithi064.vriyor (mgr.14156) 188 : cluster [DBG] pgmap v175: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:56:59.166 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:59.165+0000 7f7fed859700 1 -- 172.21.15.71:0/132665005 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7fe80f6590 msgr2=0x7f7fe80f69b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:56:59.166 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:59.165+0000 7f7fed859700 1 --2- 172.21.15.71:0/132665005 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7fe80f6590 0x7f7fe80f69b0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 rx=0x7f7fd8009fb0 tx=0x7f7fd8009b30).stop 2024-02-20T12:56:59.166 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:59.165+0000 7f7fed859700 1 -- 172.21.15.71:0/132665005 shutdown_connections 2024-02-20T12:56:59.166 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:59.165+0000 7f7fed859700 1 --2- 172.21.15.71:0/132665005 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7fe80f6590 0x7f7fe80f69b0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:56:59.166 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:59.165+0000 7f7fed859700 1 -- 172.21.15.71:0/132665005 >> 172.21.15.71:0/132665005 conn(0x7f7fe80f1cf0 msgr2=0x7f7fe80f4110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:56:59.166 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:59.165+0000 7f7fed859700 1 -- 172.21.15.71:0/132665005 shutdown_connections 2024-02-20T12:56:59.167 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:59.166+0000 7f7fed859700 1 -- 172.21.15.71:0/132665005 wait complete. 2024-02-20T12:56:59.168 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:59.166+0000 7f7fed859700 1 Processor -- start 2024-02-20T12:56:59.168 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:59.166+0000 7f7fed859700 1 -- start start 2024-02-20T12:56:59.168 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:59.166+0000 7f7fed859700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7fe80f6590 0x7f7fe806ba80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:56:59.169 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:59.166+0000 7f7fed859700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fe8069f00 con 0x7f7fe80f6590 2024-02-20T12:56:59.169 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:59.166+0000 7f7fe6ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7fe80f6590 0x7f7fe806ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:56:59.169 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:59.166+0000 7f7fe6ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7fe80f6590 0x7f7fe806ba80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.71:48318/0 (socket says 172.21.15.71:48318) 2024-02-20T12:56:59.169 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:59.166+0000 7f7fe6ffd700 1 -- 172.21.15.71:0/1465344881 learned_addr learned my addr 172.21.15.71:0/1465344881 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-02-20T12:56:59.169 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:59.167+0000 7f7fe6ffd700 1 -- 172.21.15.71:0/1465344881 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fd8005040 con 0x7f7fe80f6590 2024-02-20T12:56:59.169 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:59.167+0000 7f7fe6ffd700 1 --2- 172.21.15.71:0/1465344881 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7fe80f6590 0x7f7fe806ba80 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 rx=0x7f7fd80089f0 tx=0x7f7fd800b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:56:59.169 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:59.167+0000 7f7fcf7fe700 1 -- 172.21.15.71:0/1465344881 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7fd8009320 con 0x7f7fe80f6590 2024-02-20T12:56:59.169 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:59.167+0000 7f7fed859700 1 -- 172.21.15.71:0/1465344881 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7fe806a0a0 con 0x7f7fe80f6590 2024-02-20T12:56:59.169 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:59.167+0000 7f7fcf7fe700 1 -- 172.21.15.71:0/1465344881 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(25 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f7fd8013ec0 con 0x7f7fe80f6590 2024-02-20T12:56:59.169 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:59.167+0000 7f7fcf7fe700 1 -- 172.21.15.71:0/1465344881 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7fd8014d30 con 0x7f7fe80f6590 2024-02-20T12:56:59.169 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:59.167+0000 7f7fed859700 1 -- 172.21.15.71:0/1465344881 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fe806a530 con 0x7f7fe80f6590 2024-02-20T12:56:59.169 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:59.168+0000 7f7fcf7fe700 1 -- 172.21.15.71:0/1465344881 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 37151+0+0 (secure 0 0 0) 0x7f7fd80095f0 con 0x7f7fe80f6590 2024-02-20T12:56:59.169 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:59.168+0000 7f7fcf7fe700 1 --2- 172.21.15.71:0/1465344881 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f7fd00310e0 0x7f7fd00335a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:56:59.169 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:59.168+0000 7f7fed859700 1 -- 172.21.15.71:0/1465344881 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7fe80591e0 con 0x7f7fe80f6590 2024-02-20T12:56:59.170 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:59.168+0000 7f7fcf7fe700 1 -- 172.21.15.71:0/1465344881 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1065+0+0 (secure 0 0 0) 0x7f7fd80424a0 con 0x7f7fe80f6590 2024-02-20T12:56:59.170 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:59.168+0000 7f7fe67fc700 1 --2- 172.21.15.71:0/1465344881 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f7fd00310e0 0x7f7fd00335a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:56:59.170 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:59.169+0000 7f7fe67fc700 1 --2- 172.21.15.71:0/1465344881 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f7fd00310e0 0x7f7fd00335a0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 rx=0x7f7fdc0096a0 tx=0x7f7fdc006b40).ready entity=mgr.14156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:56:59.173 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:59.173+0000 7f7fcf7fe700 1 -- 172.21.15.71:0/1465344881 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+153237 (secure 0 0 0) 0x7f7fd801dd70 con 0x7f7fe80f6590 2024-02-20T12:56:59.380 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:59.380+0000 7f7fed859700 1 -- 172.21.15.71:0/1465344881 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f7fe80fc080 con 0x7f7fe80f6590 2024-02-20T12:56:59.381 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:59.381+0000 7f7fcf7fe700 1 -- 172.21.15.71:0/1465344881 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7f7fd801d380 con 0x7f7fe80f6590 2024-02-20T12:56:59.382 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-20T12:56:59.382 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":1,"fsid":"680ee126-cfee-11ee-95bb-87774f69a715","modified":"2024-02-20T12:49:44.699059Z","created":"2024-02-20T12:49:44.699059Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi064","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:3300","nonce":0},{"type":"v1","addr":"172.21.15.64:6789","nonce":0}]},"addr":"172.21.15.64:6789/0","public_addr":"172.21.15.64:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-20T12:56:59.383 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:59.383+0000 7f7fed859700 1 -- 172.21.15.71:0/1465344881 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f7fd00310e0 msgr2=0x7f7fd00335a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:56:59.383 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:59.383+0000 7f7fed859700 1 --2- 172.21.15.71:0/1465344881 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f7fd00310e0 0x7f7fd00335a0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 rx=0x7f7fdc0096a0 tx=0x7f7fdc006b40).stop 2024-02-20T12:56:59.383 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:59.383+0000 7f7fed859700 1 -- 172.21.15.71:0/1465344881 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7fe80f6590 msgr2=0x7f7fe806ba80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:56:59.383 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:59.383+0000 7f7fed859700 1 --2- 172.21.15.71:0/1465344881 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7fe80f6590 0x7f7fe806ba80 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 rx=0x7f7fd80089f0 tx=0x7f7fd800b040).stop 2024-02-20T12:56:59.383 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:59.383+0000 7f7fed859700 1 -- 172.21.15.71:0/1465344881 shutdown_connections 2024-02-20T12:56:59.383 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:59.383+0000 7f7fed859700 1 --2- 172.21.15.71:0/1465344881 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f7fd00310e0 0x7f7fd00335a0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:56:59.384 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:59.383+0000 7f7fed859700 1 --2- 172.21.15.71:0/1465344881 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7fe80f6590 0x7f7fe806ba80 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:56:59.384 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:59.383+0000 7f7fed859700 1 -- 172.21.15.71:0/1465344881 >> 172.21.15.71:0/1465344881 conn(0x7f7fe80f1cf0 msgr2=0x7f7fe80f27e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:56:59.384 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:59.383+0000 7f7fed859700 1 -- 172.21.15.71:0/1465344881 shutdown_connections 2024-02-20T12:56:59.384 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:56:59.383+0000 7f7fed859700 1 -- 172.21.15.71:0/1465344881 wait complete. 2024-02-20T12:56:59.384 INFO:teuthology.orchestra.run.smithi071.stderr:dumped monmap epoch 1 2024-02-20T12:56:59.450 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:56:59 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:56:57.759724+0000 mgr.smithi064.vriyor (mgr.14156) 189 : cluster [DBG] pgmap v176: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:57:00.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:57:00 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:56:59.381600+0000 mon.smithi064 (mon.0) 215 : audit [DBG] from='client.? 172.21.15.71:0/1465344881' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-20T12:57:01.099 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-20T12:57:01.099 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/daemon-base:latest-pacific shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 680ee126-cfee-11ee-95bb-87774f69a715 -- ceph mon dump -f json 2024-02-20T12:57:01.555 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:57:01 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:56:59.760056+0000 mgr.smithi064.vriyor (mgr.14156) 190 : cluster [DBG] pgmap v177: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:57:02.812 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:57:02 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:57:01.498135+0000 mon.smithi064 (mon.0) 216 : audit [INF] from='mgr.14156 172.21.15.64:0/3284974785' entity='mgr.smithi064.vriyor' 2024-02-20T12:57:02.813 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:57:02 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cephadm 2024-02-20T12:57:01.500151+0000 mgr.smithi064.vriyor (mgr.14156) 191 : cephadm [INF] Deploying daemon node-exporter.smithi064 on smithi064 2024-02-20T12:57:02.813 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:57:02 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:57:01.760349+0000 mgr.smithi064.vriyor (mgr.14156) 192 : cluster [DBG] pgmap v178: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:57:03.137 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:03.135+0000 7f1c1762b700 1 -- 172.21.15.71:0/938236448 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1c100f5480 msgr2=0x7f1c100f58a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:57:03.138 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:03.135+0000 7f1c1762b700 1 --2- 172.21.15.71:0/938236448 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1c100f5480 0x7f1c100f58a0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 rx=0x7f1c08009fb0 tx=0x7f1c08009b30).stop 2024-02-20T12:57:03.138 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:03.135+0000 7f1c1762b700 1 -- 172.21.15.71:0/938236448 shutdown_connections 2024-02-20T12:57:03.138 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:03.135+0000 7f1c1762b700 1 --2- 172.21.15.71:0/938236448 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1c100f5480 0x7f1c100f58a0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:57:03.138 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:03.135+0000 7f1c1762b700 1 -- 172.21.15.71:0/938236448 >> 172.21.15.71:0/938236448 conn(0x7f1c100f0be0 msgr2=0x7f1c100f3000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:57:03.138 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:03.135+0000 7f1c1762b700 1 -- 172.21.15.71:0/938236448 shutdown_connections 2024-02-20T12:57:03.138 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:03.135+0000 7f1c1762b700 1 -- 172.21.15.71:0/938236448 wait complete. 2024-02-20T12:57:03.138 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:03.136+0000 7f1c1762b700 1 Processor -- start 2024-02-20T12:57:03.138 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:03.136+0000 7f1c1762b700 1 -- start start 2024-02-20T12:57:03.138 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:03.136+0000 7f1c1762b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1c100f5480 0x7f1c100fd460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:57:03.138 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:03.136+0000 7f1c1762b700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c100fb8e0 con 0x7f1c100f5480 2024-02-20T12:57:03.139 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:03.136+0000 7f1c153c7700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1c100f5480 0x7f1c100fd460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:57:03.139 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:03.136+0000 7f1c153c7700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1c100f5480 0x7f1c100fd460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.71:48326/0 (socket says 172.21.15.71:48326) 2024-02-20T12:57:03.139 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:03.136+0000 7f1c153c7700 1 -- 172.21.15.71:0/3931093846 learned_addr learned my addr 172.21.15.71:0/3931093846 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-02-20T12:57:03.139 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:03.136+0000 7f1c153c7700 1 -- 172.21.15.71:0/3931093846 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c08005040 con 0x7f1c100f5480 2024-02-20T12:57:03.139 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:03.136+0000 7f1c153c7700 1 --2- 172.21.15.71:0/3931093846 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1c100f5480 0x7f1c100fd460 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 rx=0x7f1c08000c00 tx=0x7f1c0800b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:57:03.139 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:03.137+0000 7f1c0dffb700 1 -- 172.21.15.71:0/3931093846 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1c080095f0 con 0x7f1c100f5480 2024-02-20T12:57:03.139 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:03.137+0000 7f1c0dffb700 1 -- 172.21.15.71:0/3931093846 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(25 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f1c08007e50 con 0x7f1c100f5480 2024-02-20T12:57:03.139 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:03.137+0000 7f1c1762b700 1 -- 172.21.15.71:0/3931093846 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c100fba80 con 0x7f1c100f5480 2024-02-20T12:57:03.139 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:03.137+0000 7f1c0dffb700 1 -- 172.21.15.71:0/3931093846 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1c08014e70 con 0x7f1c100f5480 2024-02-20T12:57:03.139 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:03.137+0000 7f1c1762b700 1 -- 172.21.15.71:0/3931093846 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c100fbf10 con 0x7f1c100f5480 2024-02-20T12:57:03.140 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:03.138+0000 7f1c0dffb700 1 -- 172.21.15.71:0/3931093846 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 37151+0+0 (secure 0 0 0) 0x7f1c08025070 con 0x7f1c100f5480 2024-02-20T12:57:03.141 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:03.138+0000 7f1c1762b700 1 -- 172.21.15.71:0/3931093846 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c100591e0 con 0x7f1c100f5480 2024-02-20T12:57:03.141 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:03.138+0000 7f1c0dffb700 1 --2- 172.21.15.71:0/3931093846 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f1bf8031090 0x7f1bf8033550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:57:03.141 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:03.138+0000 7f1c0dffb700 1 -- 172.21.15.71:0/3931093846 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1065+0+0 (secure 0 0 0) 0x7f1c08041d10 con 0x7f1c100f5480 2024-02-20T12:57:03.141 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:03.138+0000 7f1c14bc6700 1 --2- 172.21.15.71:0/3931093846 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f1bf8031090 0x7f1bf8033550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:57:03.141 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:03.139+0000 7f1c14bc6700 1 --2- 172.21.15.71:0/3931093846 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f1bf8031090 0x7f1bf8033550 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 rx=0x7f1c040096a0 tx=0x7f1c04006b40).ready entity=mgr.14156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:57:03.143 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:03.143+0000 7f1c0dffb700 1 -- 172.21.15.71:0/3931093846 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+153237 (secure 0 0 0) 0x7f1c0800f4e0 con 0x7f1c100f5480 2024-02-20T12:57:03.351 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:03.351+0000 7f1c1762b700 1 -- 172.21.15.71:0/3931093846 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f1c10045bb0 con 0x7f1c100f5480 2024-02-20T12:57:03.352 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:03.352+0000 7f1c0dffb700 1 -- 172.21.15.71:0/3931093846 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7f1c0800f300 con 0x7f1c100f5480 2024-02-20T12:57:03.352 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-20T12:57:03.352 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":1,"fsid":"680ee126-cfee-11ee-95bb-87774f69a715","modified":"2024-02-20T12:49:44.699059Z","created":"2024-02-20T12:49:44.699059Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi064","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:3300","nonce":0},{"type":"v1","addr":"172.21.15.64:6789","nonce":0}]},"addr":"172.21.15.64:6789/0","public_addr":"172.21.15.64:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-20T12:57:03.354 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:03.353+0000 7f1c1762b700 1 -- 172.21.15.71:0/3931093846 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f1bf8031090 msgr2=0x7f1bf8033550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:57:03.354 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:03.353+0000 7f1c1762b700 1 --2- 172.21.15.71:0/3931093846 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f1bf8031090 0x7f1bf8033550 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 rx=0x7f1c040096a0 tx=0x7f1c04006b40).stop 2024-02-20T12:57:03.354 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:03.353+0000 7f1c1762b700 1 -- 172.21.15.71:0/3931093846 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1c100f5480 msgr2=0x7f1c100fd460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:57:03.354 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:03.353+0000 7f1c1762b700 1 --2- 172.21.15.71:0/3931093846 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1c100f5480 0x7f1c100fd460 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 rx=0x7f1c08000c00 tx=0x7f1c0800b040).stop 2024-02-20T12:57:03.354 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:03.353+0000 7f1c1762b700 1 -- 172.21.15.71:0/3931093846 shutdown_connections 2024-02-20T12:57:03.354 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:03.353+0000 7f1c1762b700 1 --2- 172.21.15.71:0/3931093846 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f1bf8031090 0x7f1bf8033550 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:57:03.354 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:03.353+0000 7f1c1762b700 1 --2- 172.21.15.71:0/3931093846 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1c100f5480 0x7f1c100fd460 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:57:03.354 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:03.353+0000 7f1c1762b700 1 -- 172.21.15.71:0/3931093846 >> 172.21.15.71:0/3931093846 conn(0x7f1c100f0be0 msgr2=0x7f1c1005ff40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:57:03.355 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:03.354+0000 7f1c1762b700 1 -- 172.21.15.71:0/3931093846 shutdown_connections 2024-02-20T12:57:03.355 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:03.354+0000 7f1c1762b700 1 -- 172.21.15.71:0/3931093846 wait complete. 2024-02-20T12:57:03.355 INFO:teuthology.orchestra.run.smithi071.stderr:dumped monmap epoch 1 2024-02-20T12:57:03.812 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:57:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:57:03.352448+0000 mon.smithi064 (mon.0) 217 : audit [DBG] from='client.? 172.21.15.71:0/3931093846' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-20T12:57:05.162 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-20T12:57:05.163 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/daemon-base:latest-pacific shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 680ee126-cfee-11ee-95bb-87774f69a715 -- ceph mon dump -f json 2024-02-20T12:57:05.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:57:04 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:57:03.760760+0000 mgr.smithi064.vriyor (mgr.14156) 193 : cluster [DBG] pgmap v179: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:57:05.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:57:04 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:57:03.854251+0000 mon.smithi064 (mon.0) 218 : audit [INF] from='mgr.14156 172.21.15.64:0/3284974785' entity='mgr.smithi064.vriyor' 2024-02-20T12:57:07.122 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:07.120+0000 7fed7cf62700 1 -- 172.21.15.71:0/3424444709 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fed780f6590 msgr2=0x7fed780f69b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:57:07.122 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:07.120+0000 7fed7cf62700 1 --2- 172.21.15.71:0/3424444709 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fed780f6590 0x7fed780f69b0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 rx=0x7fed6c004760 tx=0x7fed6c009ad0).stop 2024-02-20T12:57:07.123 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:07.120+0000 7fed7cf62700 1 -- 172.21.15.71:0/3424444709 shutdown_connections 2024-02-20T12:57:07.123 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:07.120+0000 7fed7cf62700 1 --2- 172.21.15.71:0/3424444709 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fed780f6590 0x7fed780f69b0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:57:07.123 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:07.120+0000 7fed7cf62700 1 -- 172.21.15.71:0/3424444709 >> 172.21.15.71:0/3424444709 conn(0x7fed780f1cd0 msgr2=0x7fed780f4110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:57:07.123 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:07.120+0000 7fed7cf62700 1 -- 172.21.15.71:0/3424444709 shutdown_connections 2024-02-20T12:57:07.123 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:07.120+0000 7fed7cf62700 1 -- 172.21.15.71:0/3424444709 wait complete. 2024-02-20T12:57:07.123 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:07.121+0000 7fed7cf62700 1 Processor -- start 2024-02-20T12:57:07.123 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:07.121+0000 7fed7cf62700 1 -- start start 2024-02-20T12:57:07.123 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:07.121+0000 7fed7cf62700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fed780f6590 0x7fed7806ba80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:57:07.123 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:07.121+0000 7fed7cf62700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed78069f00 con 0x7fed780f6590 2024-02-20T12:57:07.123 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:07.121+0000 7fed7659c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fed780f6590 0x7fed7806ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:57:07.123 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:07.121+0000 7fed7659c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fed780f6590 0x7fed7806ba80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.71:48342/0 (socket says 172.21.15.71:48342) 2024-02-20T12:57:07.124 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:07.121+0000 7fed7659c700 1 -- 172.21.15.71:0/2944980171 learned_addr learned my addr 172.21.15.71:0/2944980171 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-02-20T12:57:07.124 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:07.121+0000 7fed7659c700 1 -- 172.21.15.71:0/2944980171 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed6c009d70 con 0x7fed780f6590 2024-02-20T12:57:07.124 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:07.121+0000 7fed7659c700 1 --2- 172.21.15.71:0/2944980171 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fed780f6590 0x7fed7806ba80 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 rx=0x7fed6c004170 tx=0x7fed6c009ad0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:57:07.124 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:07.122+0000 7fed66ffd700 1 -- 172.21.15.71:0/2944980171 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fed6c00e040 con 0x7fed780f6590 2024-02-20T12:57:07.124 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:07.122+0000 7fed66ffd700 1 -- 172.21.15.71:0/2944980171 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(25 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7fed6c008ef0 con 0x7fed780f6590 2024-02-20T12:57:07.124 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:07.122+0000 7fed7cf62700 1 -- 172.21.15.71:0/2944980171 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed7806a0a0 con 0x7fed780f6590 2024-02-20T12:57:07.124 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:07.122+0000 7fed66ffd700 1 -- 172.21.15.71:0/2944980171 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fed6c014c90 con 0x7fed780f6590 2024-02-20T12:57:07.125 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:07.122+0000 7fed7cf62700 1 -- 172.21.15.71:0/2944980171 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed7806a530 con 0x7fed780f6590 2024-02-20T12:57:07.126 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:07.122+0000 7fed66ffd700 1 -- 172.21.15.71:0/2944980171 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 37151+0+0 (secure 0 0 0) 0x7fed6c014420 con 0x7fed780f6590 2024-02-20T12:57:07.126 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:07.123+0000 7fed7cf62700 1 -- 172.21.15.71:0/2944980171 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed780591e0 con 0x7fed780f6590 2024-02-20T12:57:07.126 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:07.123+0000 7fed66ffd700 1 --2- 172.21.15.71:0/2944980171 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fed60031090 0x7fed60033550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:57:07.127 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:07.123+0000 7fed66ffd700 1 -- 172.21.15.71:0/2944980171 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1065+0+0 (secure 0 0 0) 0x7fed6c042220 con 0x7fed780f6590 2024-02-20T12:57:07.127 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:07.123+0000 7fed75d9b700 1 --2- 172.21.15.71:0/2944980171 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fed60031090 0x7fed60033550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:57:07.127 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:07.123+0000 7fed75d9b700 1 --2- 172.21.15.71:0/2944980171 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fed60031090 0x7fed60033550 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 rx=0x7fed680096a0 tx=0x7fed68006b40).ready entity=mgr.14156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:57:07.128 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:07.127+0000 7fed66ffd700 1 -- 172.21.15.71:0/2944980171 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+153237 (secure 0 0 0) 0x7fed6c012d00 con 0x7fed780f6590 2024-02-20T12:57:07.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:57:06 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:57:05.761184+0000 mgr.smithi064.vriyor (mgr.14156) 194 : cluster [DBG] pgmap v180: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:57:07.334 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:07.334+0000 7fed7cf62700 1 -- 172.21.15.71:0/2944980171 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fed780fc080 con 0x7fed780f6590 2024-02-20T12:57:07.335 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:07.335+0000 7fed66ffd700 1 -- 172.21.15.71:0/2944980171 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7fed6c012b20 con 0x7fed780f6590 2024-02-20T12:57:07.335 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-20T12:57:07.335 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":1,"fsid":"680ee126-cfee-11ee-95bb-87774f69a715","modified":"2024-02-20T12:49:44.699059Z","created":"2024-02-20T12:49:44.699059Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi064","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:3300","nonce":0},{"type":"v1","addr":"172.21.15.64:6789","nonce":0}]},"addr":"172.21.15.64:6789/0","public_addr":"172.21.15.64:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-20T12:57:07.337 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:07.336+0000 7fed7cf62700 1 -- 172.21.15.71:0/2944980171 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fed60031090 msgr2=0x7fed60033550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:57:07.337 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:07.336+0000 7fed7cf62700 1 --2- 172.21.15.71:0/2944980171 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fed60031090 0x7fed60033550 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 rx=0x7fed680096a0 tx=0x7fed68006b40).stop 2024-02-20T12:57:07.337 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:07.336+0000 7fed7cf62700 1 -- 172.21.15.71:0/2944980171 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fed780f6590 msgr2=0x7fed7806ba80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:57:07.337 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:07.336+0000 7fed7cf62700 1 --2- 172.21.15.71:0/2944980171 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fed780f6590 0x7fed7806ba80 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 rx=0x7fed6c004170 tx=0x7fed6c009ad0).stop 2024-02-20T12:57:07.337 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:07.336+0000 7fed7cf62700 1 -- 172.21.15.71:0/2944980171 shutdown_connections 2024-02-20T12:57:07.337 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:07.336+0000 7fed7cf62700 1 --2- 172.21.15.71:0/2944980171 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fed60031090 0x7fed60033550 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:57:07.338 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:07.336+0000 7fed7cf62700 1 --2- 172.21.15.71:0/2944980171 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fed780f6590 0x7fed7806ba80 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:57:07.338 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:07.336+0000 7fed7cf62700 1 -- 172.21.15.71:0/2944980171 >> 172.21.15.71:0/2944980171 conn(0x7fed780f1cd0 msgr2=0x7fed780f27c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:57:07.338 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:07.336+0000 7fed7cf62700 1 -- 172.21.15.71:0/2944980171 shutdown_connections 2024-02-20T12:57:07.338 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:07.336+0000 7fed7cf62700 1 -- 172.21.15.71:0/2944980171 wait complete. 2024-02-20T12:57:07.338 INFO:teuthology.orchestra.run.smithi071.stderr:dumped monmap epoch 1 2024-02-20T12:57:08.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:57:07 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:57:07.335476+0000 mon.smithi064 (mon.0) 219 : audit [DBG] from='client.? 172.21.15.71:0/2944980171' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-20T12:57:08.949 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-20T12:57:08.949 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/daemon-base:latest-pacific shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 680ee126-cfee-11ee-95bb-87774f69a715 -- ceph mon dump -f json 2024-02-20T12:57:09.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:57:08 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:57:07.761517+0000 mgr.smithi064.vriyor (mgr.14156) 195 : cluster [DBG] pgmap v181: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:57:10.843 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:10.842+0000 7febe9be1700 1 -- 172.21.15.71:0/1213066963 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7febe4060360 msgr2=0x7febe40f9160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:57:10.843 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:10.842+0000 7febe9be1700 1 --2- 172.21.15.71:0/1213066963 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7febe4060360 0x7febe40f9160 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 rx=0x7febd4009960 tx=0x7febd4004d10).stop 2024-02-20T12:57:10.843 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:10.842+0000 7febe9be1700 1 -- 172.21.15.71:0/1213066963 shutdown_connections 2024-02-20T12:57:10.843 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:10.842+0000 7febe9be1700 1 --2- 172.21.15.71:0/1213066963 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7febe4060360 0x7febe40f9160 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:57:10.843 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:10.842+0000 7febe9be1700 1 -- 172.21.15.71:0/1213066963 >> 172.21.15.71:0/1213066963 conn(0x7febe40f0be0 msgr2=0x7febe40f3000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:57:10.843 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:10.842+0000 7febe9be1700 1 -- 172.21.15.71:0/1213066963 shutdown_connections 2024-02-20T12:57:10.843 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:10.842+0000 7febe9be1700 1 -- 172.21.15.71:0/1213066963 wait complete. 2024-02-20T12:57:10.843 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:10.842+0000 7febe9be1700 1 Processor -- start 2024-02-20T12:57:10.843 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:10.843+0000 7febe9be1700 1 -- start start 2024-02-20T12:57:10.845 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:10.843+0000 7febe9be1700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7febe4060360 0x7febe4106750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:57:10.845 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:10.843+0000 7febe9be1700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febe4106c40 con 0x7febe4060360 2024-02-20T12:57:10.845 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:10.843+0000 7febe37fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7febe4060360 0x7febe4106750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:57:10.845 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:10.843+0000 7febe37fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7febe4060360 0x7febe4106750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.71:40432/0 (socket says 172.21.15.71:40432) 2024-02-20T12:57:10.845 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:10.843+0000 7febe37fe700 1 -- 172.21.15.71:0/1286119939 learned_addr learned my addr 172.21.15.71:0/1286119939 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-02-20T12:57:10.845 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:10.843+0000 7febe37fe700 1 -- 172.21.15.71:0/1286119939 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febd4009640 con 0x7febe4060360 2024-02-20T12:57:10.845 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:10.843+0000 7febe37fe700 1 --2- 172.21.15.71:0/1286119939 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7febe4060360 0x7febe4106750 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 rx=0x7febd4009610 tx=0x7febd4004d10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:57:10.845 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:10.843+0000 7febdbfff700 1 -- 172.21.15.71:0/1286119939 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7febd4018070 con 0x7febe4060360 2024-02-20T12:57:10.845 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:10.844+0000 7febe9be1700 1 -- 172.21.15.71:0/1286119939 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7febe4106e40 con 0x7febe4060360 2024-02-20T12:57:10.846 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:10.844+0000 7febdbfff700 1 -- 172.21.15.71:0/1286119939 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(25 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7febd400b7e0 con 0x7febe4060360 2024-02-20T12:57:10.846 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:10.844+0000 7febdbfff700 1 -- 172.21.15.71:0/1286119939 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7febd401d420 con 0x7febe4060360 2024-02-20T12:57:10.846 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:10.844+0000 7febe9be1700 1 -- 172.21.15.71:0/1286119939 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febe41072d0 con 0x7febe4060360 2024-02-20T12:57:10.846 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:10.844+0000 7febdbfff700 1 -- 172.21.15.71:0/1286119939 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 37151+0+0 (secure 0 0 0) 0x7febd4024060 con 0x7febe4060360 2024-02-20T12:57:10.846 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:10.844+0000 7febe9be1700 1 -- 172.21.15.71:0/1286119939 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7febe4045bb0 con 0x7febe4060360 2024-02-20T12:57:10.846 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:10.845+0000 7febdbfff700 1 --2- 172.21.15.71:0/1286119939 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7febd0031090 0x7febd0033550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:57:10.846 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:10.845+0000 7febdbfff700 1 -- 172.21.15.71:0/1286119939 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1065+0+0 (secure 0 0 0) 0x7febd40426e0 con 0x7febe4060360 2024-02-20T12:57:10.846 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:10.845+0000 7febe2ffd700 1 --2- 172.21.15.71:0/1286119939 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7febd0031090 0x7febd0033550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:57:10.846 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:10.845+0000 7febe2ffd700 1 --2- 172.21.15.71:0/1286119939 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7febd0031090 0x7febd0033550 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 rx=0x7febcc0096a0 tx=0x7febcc006b40).ready entity=mgr.14156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:57:10.850 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:10.849+0000 7febdbfff700 1 -- 172.21.15.71:0/1286119939 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+153237 (secure 0 0 0) 0x7febd4011b60 con 0x7febe4060360 2024-02-20T12:57:11.056 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:11.056+0000 7febe9be1700 1 -- 172.21.15.71:0/1286119939 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7febe4108120 con 0x7febe4060360 2024-02-20T12:57:11.057 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:11.057+0000 7febdbfff700 1 -- 172.21.15.71:0/1286119939 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7febd4011930 con 0x7febe4060360 2024-02-20T12:57:11.057 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-20T12:57:11.057 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":1,"fsid":"680ee126-cfee-11ee-95bb-87774f69a715","modified":"2024-02-20T12:49:44.699059Z","created":"2024-02-20T12:49:44.699059Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi064","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:3300","nonce":0},{"type":"v1","addr":"172.21.15.64:6789","nonce":0}]},"addr":"172.21.15.64:6789/0","public_addr":"172.21.15.64:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-20T12:57:11.059 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:11.058+0000 7febe9be1700 1 -- 172.21.15.71:0/1286119939 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7febd0031090 msgr2=0x7febd0033550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:57:11.059 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:11.058+0000 7febe9be1700 1 --2- 172.21.15.71:0/1286119939 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7febd0031090 0x7febd0033550 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 rx=0x7febcc0096a0 tx=0x7febcc006b40).stop 2024-02-20T12:57:11.059 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:11.058+0000 7febe9be1700 1 -- 172.21.15.71:0/1286119939 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7febe4060360 msgr2=0x7febe4106750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:57:11.059 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:11.058+0000 7febe9be1700 1 --2- 172.21.15.71:0/1286119939 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7febe4060360 0x7febe4106750 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 rx=0x7febd4009610 tx=0x7febd4004d10).stop 2024-02-20T12:57:11.059 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:11.058+0000 7febe9be1700 1 -- 172.21.15.71:0/1286119939 shutdown_connections 2024-02-20T12:57:11.059 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:11.058+0000 7febe9be1700 1 --2- 172.21.15.71:0/1286119939 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7febd0031090 0x7febd0033550 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:57:11.059 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:11.058+0000 7febe9be1700 1 --2- 172.21.15.71:0/1286119939 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7febe4060360 0x7febe4106750 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:57:11.059 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:11.058+0000 7febe9be1700 1 -- 172.21.15.71:0/1286119939 >> 172.21.15.71:0/1286119939 conn(0x7febe40f0be0 msgr2=0x7febe40f1660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:57:11.059 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:11.059+0000 7febe9be1700 1 -- 172.21.15.71:0/1286119939 shutdown_connections 2024-02-20T12:57:11.060 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:11.059+0000 7febe9be1700 1 -- 172.21.15.71:0/1286119939 wait complete. 2024-02-20T12:57:11.060 INFO:teuthology.orchestra.run.smithi071.stderr:dumped monmap epoch 1 2024-02-20T12:57:11.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:57:10 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:57:09.761902+0000 mgr.smithi064.vriyor (mgr.14156) 196 : cluster [DBG] pgmap v182: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:57:12.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:57:11 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:57:11.057660+0000 mon.smithi064 (mon.0) 220 : audit [DBG] from='client.? 172.21.15.71:0/1286119939' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-20T12:57:12.875 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-20T12:57:12.922 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/daemon-base:latest-pacific shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 680ee126-cfee-11ee-95bb-87774f69a715 -- ceph mon dump -f json 2024-02-20T12:57:13.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:57:12 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:57:11.762214+0000 mgr.smithi064.vriyor (mgr.14156) 197 : cluster [DBG] pgmap v183: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:57:14.670 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:14.668+0000 7fa2ff3ab700 1 -- 172.21.15.71:0/2321727306 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa2f80f5480 msgr2=0x7fa2f80f58a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:57:14.670 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:14.668+0000 7fa2ff3ab700 1 --2- 172.21.15.71:0/2321727306 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa2f80f5480 0x7fa2f80f58a0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 rx=0x7fa2f4009fb0 tx=0x7fa2f4009b30).stop 2024-02-20T12:57:14.670 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:14.668+0000 7fa2ff3ab700 1 -- 172.21.15.71:0/2321727306 shutdown_connections 2024-02-20T12:57:14.670 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:14.668+0000 7fa2ff3ab700 1 --2- 172.21.15.71:0/2321727306 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa2f80f5480 0x7fa2f80f58a0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:57:14.670 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:14.668+0000 7fa2ff3ab700 1 -- 172.21.15.71:0/2321727306 >> 172.21.15.71:0/2321727306 conn(0x7fa2f80f0be0 msgr2=0x7fa2f80f3000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:57:14.670 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:14.668+0000 7fa2ff3ab700 1 -- 172.21.15.71:0/2321727306 shutdown_connections 2024-02-20T12:57:14.670 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:14.668+0000 7fa2ff3ab700 1 -- 172.21.15.71:0/2321727306 wait complete. 2024-02-20T12:57:14.670 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:14.668+0000 7fa2ff3ab700 1 Processor -- start 2024-02-20T12:57:14.671 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:14.668+0000 7fa2ff3ab700 1 -- start start 2024-02-20T12:57:14.671 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:14.669+0000 7fa2ff3ab700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa2f80f5480 0x7fa2f81056f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:57:14.671 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:14.669+0000 7fa2ff3ab700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2f8105be0 con 0x7fa2f80f5480 2024-02-20T12:57:14.671 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:14.669+0000 7fa2fd147700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa2f80f5480 0x7fa2f81056f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:57:14.671 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:14.669+0000 7fa2fd147700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa2f80f5480 0x7fa2f81056f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.71:40438/0 (socket says 172.21.15.71:40438) 2024-02-20T12:57:14.671 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:14.669+0000 7fa2fd147700 1 -- 172.21.15.71:0/1214411818 learned_addr learned my addr 172.21.15.71:0/1214411818 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-02-20T12:57:14.671 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:14.669+0000 7fa2fd147700 1 -- 172.21.15.71:0/1214411818 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2f4005040 con 0x7fa2f80f5480 2024-02-20T12:57:14.671 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:14.669+0000 7fa2fd147700 1 --2- 172.21.15.71:0/1214411818 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa2f80f5480 0x7fa2f81056f0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 rx=0x7fa2f40089f0 tx=0x7fa2f400b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:57:14.671 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:14.669+0000 7fa2edffb700 1 -- 172.21.15.71:0/1214411818 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa2f4009320 con 0x7fa2f80f5480 2024-02-20T12:57:14.671 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:14.669+0000 7fa2edffb700 1 -- 172.21.15.71:0/1214411818 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(25 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7fa2f4013ec0 con 0x7fa2f80f5480 2024-02-20T12:57:14.671 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:14.669+0000 7fa2edffb700 1 -- 172.21.15.71:0/1214411818 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa2f4014d30 con 0x7fa2f80f5480 2024-02-20T12:57:14.671 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:14.669+0000 7fa2ff3ab700 1 -- 172.21.15.71:0/1214411818 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2f8105de0 con 0x7fa2f80f5480 2024-02-20T12:57:14.671 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:14.670+0000 7fa2ff3ab700 1 -- 172.21.15.71:0/1214411818 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2f8106270 con 0x7fa2f80f5480 2024-02-20T12:57:14.672 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:14.670+0000 7fa2edffb700 1 -- 172.21.15.71:0/1214411818 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 37151+0+0 (secure 0 0 0) 0x7fa2f40095f0 con 0x7fa2f80f5480 2024-02-20T12:57:14.672 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:14.671+0000 7fa2edffb700 1 --2- 172.21.15.71:0/1214411818 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fa2e4031090 0x7fa2e4033550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:57:14.672 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:14.671+0000 7fa2ff3ab700 1 -- 172.21.15.71:0/1214411818 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2f8045bb0 con 0x7fa2f80f5480 2024-02-20T12:57:14.672 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:14.671+0000 7fa2edffb700 1 -- 172.21.15.71:0/1214411818 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1065+0+0 (secure 0 0 0) 0x7fa2f40423e0 con 0x7fa2f80f5480 2024-02-20T12:57:14.672 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:14.671+0000 7fa2fc946700 1 --2- 172.21.15.71:0/1214411818 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fa2e4031090 0x7fa2e4033550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:57:14.673 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:14.671+0000 7fa2fc946700 1 --2- 172.21.15.71:0/1214411818 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fa2e4031090 0x7fa2e4033550 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 rx=0x7fa2e80096a0 tx=0x7fa2e8006b40).ready entity=mgr.14156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:57:14.676 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:14.676+0000 7fa2edffb700 1 -- 172.21.15.71:0/1214411818 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+153237 (secure 0 0 0) 0x7fa2f401d7f0 con 0x7fa2f80f5480 2024-02-20T12:57:14.884 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:14.883+0000 7fa2ff3ab700 1 -- 172.21.15.71:0/1214411818 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fa2f80fb060 con 0x7fa2f80f5480 2024-02-20T12:57:14.884 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:14.884+0000 7fa2edffb700 1 -- 172.21.15.71:0/1214411818 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7fa2f4039090 con 0x7fa2f80f5480 2024-02-20T12:57:14.886 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-20T12:57:14.886 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":1,"fsid":"680ee126-cfee-11ee-95bb-87774f69a715","modified":"2024-02-20T12:49:44.699059Z","created":"2024-02-20T12:49:44.699059Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi064","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:3300","nonce":0},{"type":"v1","addr":"172.21.15.64:6789","nonce":0}]},"addr":"172.21.15.64:6789/0","public_addr":"172.21.15.64:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-20T12:57:14.888 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:14.885+0000 7fa2ff3ab700 1 -- 172.21.15.71:0/1214411818 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fa2e4031090 msgr2=0x7fa2e4033550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:57:14.888 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:14.885+0000 7fa2ff3ab700 1 --2- 172.21.15.71:0/1214411818 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fa2e4031090 0x7fa2e4033550 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 rx=0x7fa2e80096a0 tx=0x7fa2e8006b40).stop 2024-02-20T12:57:14.888 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:14.885+0000 7fa2ff3ab700 1 -- 172.21.15.71:0/1214411818 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa2f80f5480 msgr2=0x7fa2f81056f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:57:14.888 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:14.885+0000 7fa2ff3ab700 1 --2- 172.21.15.71:0/1214411818 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa2f80f5480 0x7fa2f81056f0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 rx=0x7fa2f40089f0 tx=0x7fa2f400b040).stop 2024-02-20T12:57:14.888 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:14.885+0000 7fa2ff3ab700 1 -- 172.21.15.71:0/1214411818 shutdown_connections 2024-02-20T12:57:14.888 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:14.885+0000 7fa2ff3ab700 1 --2- 172.21.15.71:0/1214411818 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fa2e4031090 0x7fa2e4033550 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:57:14.889 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:14.885+0000 7fa2ff3ab700 1 --2- 172.21.15.71:0/1214411818 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa2f80f5480 0x7fa2f81056f0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:57:14.889 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:14.885+0000 7fa2ff3ab700 1 -- 172.21.15.71:0/1214411818 >> 172.21.15.71:0/1214411818 conn(0x7fa2f80f0be0 msgr2=0x7fa2f80f16d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:57:14.889 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:14.885+0000 7fa2ff3ab700 1 -- 172.21.15.71:0/1214411818 shutdown_connections 2024-02-20T12:57:14.889 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:14.885+0000 7fa2ff3ab700 1 -- 172.21.15.71:0/1214411818 wait complete. 2024-02-20T12:57:14.889 INFO:teuthology.orchestra.run.smithi071.stderr:dumped monmap epoch 1 2024-02-20T12:57:15.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:57:14 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:57:13.762529+0000 mgr.smithi064.vriyor (mgr.14156) 198 : cluster [DBG] pgmap v184: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:57:15.313 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:57:14 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:57:14.884733+0000 mon.smithi064 (mon.0) 221 : audit [DBG] from='client.? 172.21.15.71:0/1214411818' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-20T12:57:16.595 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-20T12:57:16.596 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/daemon-base:latest-pacific shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 680ee126-cfee-11ee-95bb-87774f69a715 -- ceph mon dump -f json 2024-02-20T12:57:17.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:57:16 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:57:15.762930+0000 mgr.smithi064.vriyor (mgr.14156) 199 : cluster [DBG] pgmap v185: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:57:18.443 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:18.441+0000 7fec18d8a700 1 -- 172.21.15.71:0/180278564 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fec140f5480 msgr2=0x7fec140f58a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:57:18.443 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:18.441+0000 7fec18d8a700 1 --2- 172.21.15.71:0/180278564 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fec140f5480 0x7fec140f58a0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 rx=0x7febfc009fb0 tx=0x7febfc009b30).stop 2024-02-20T12:57:18.443 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:18.441+0000 7fec18d8a700 1 -- 172.21.15.71:0/180278564 shutdown_connections 2024-02-20T12:57:18.443 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:18.441+0000 7fec18d8a700 1 --2- 172.21.15.71:0/180278564 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fec140f5480 0x7fec140f58a0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:57:18.444 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:18.441+0000 7fec18d8a700 1 -- 172.21.15.71:0/180278564 >> 172.21.15.71:0/180278564 conn(0x7fec140f0be0 msgr2=0x7fec140f3000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:57:18.444 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:18.441+0000 7fec18d8a700 1 -- 172.21.15.71:0/180278564 shutdown_connections 2024-02-20T12:57:18.444 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:18.441+0000 7fec18d8a700 1 -- 172.21.15.71:0/180278564 wait complete. 2024-02-20T12:57:18.444 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:18.442+0000 7fec18d8a700 1 Processor -- start 2024-02-20T12:57:18.444 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:18.442+0000 7fec18d8a700 1 -- start start 2024-02-20T12:57:18.444 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:18.442+0000 7fec18d8a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fec140f5480 0x7fec1410b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:57:18.444 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:18.442+0000 7fec18d8a700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec14107300 con 0x7fec140f5480 2024-02-20T12:57:18.444 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:18.442+0000 7fec1259c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fec140f5480 0x7fec1410b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:57:18.444 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:18.442+0000 7fec1259c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fec140f5480 0x7fec1410b910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.71:58684/0 (socket says 172.21.15.71:58684) 2024-02-20T12:57:18.444 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:18.442+0000 7fec1259c700 1 -- 172.21.15.71:0/1365005080 learned_addr learned my addr 172.21.15.71:0/1365005080 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-02-20T12:57:18.444 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:18.443+0000 7fec1259c700 1 -- 172.21.15.71:0/1365005080 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febfc005040 con 0x7fec140f5480 2024-02-20T12:57:18.444 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:18.443+0000 7fec1259c700 1 --2- 172.21.15.71:0/1365005080 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fec140f5480 0x7fec1410b910 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 rx=0x7febfc0089f0 tx=0x7febfc00b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:57:18.445 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:18.443+0000 7fec0affd700 1 -- 172.21.15.71:0/1365005080 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7febfc009320 con 0x7fec140f5480 2024-02-20T12:57:18.445 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:18.443+0000 7fec18d8a700 1 -- 172.21.15.71:0/1365005080 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec14060300 con 0x7fec140f5480 2024-02-20T12:57:18.445 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:18.443+0000 7fec0affd700 1 -- 172.21.15.71:0/1365005080 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(25 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7febfc013ec0 con 0x7fec140f5480 2024-02-20T12:57:18.445 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:18.443+0000 7fec0affd700 1 -- 172.21.15.71:0/1365005080 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7febfc014d30 con 0x7fec140f5480 2024-02-20T12:57:18.445 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:18.443+0000 7fec18d8a700 1 -- 172.21.15.71:0/1365005080 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec1410be00 con 0x7fec140f5480 2024-02-20T12:57:18.446 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:18.444+0000 7fec0affd700 1 -- 172.21.15.71:0/1365005080 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 37151+0+0 (secure 0 0 0) 0x7febfc0095f0 con 0x7fec140f5480 2024-02-20T12:57:18.446 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:18.444+0000 7fec18d8a700 1 -- 172.21.15.71:0/1365005080 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec14045bb0 con 0x7fec140f5480 2024-02-20T12:57:18.446 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:18.444+0000 7fec0affd700 1 --2- 172.21.15.71:0/1365005080 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fec00031090 0x7fec00033550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:57:18.446 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:18.444+0000 7fec0affd700 1 -- 172.21.15.71:0/1365005080 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1065+0+0 (secure 0 0 0) 0x7febfc042500 con 0x7fec140f5480 2024-02-20T12:57:18.446 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:18.444+0000 7fec11d9b700 1 --2- 172.21.15.71:0/1365005080 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fec00031090 0x7fec00033550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:57:18.446 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:18.445+0000 7fec11d9b700 1 --2- 172.21.15.71:0/1365005080 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fec00031090 0x7fec00033550 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 rx=0x7fec040096a0 tx=0x7fec04006b40).ready entity=mgr.14156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:57:18.449 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:18.449+0000 7fec0affd700 1 -- 172.21.15.71:0/1365005080 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+153237 (secure 0 0 0) 0x7febfc01d7f0 con 0x7fec140f5480 2024-02-20T12:57:18.657 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:18.656+0000 7fec18d8a700 1 -- 172.21.15.71:0/1365005080 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fec140faf70 con 0x7fec140f5480 2024-02-20T12:57:18.657 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:18.657+0000 7fec0affd700 1 -- 172.21.15.71:0/1365005080 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7febfc01d7f0 con 0x7fec140f5480 2024-02-20T12:57:18.658 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-20T12:57:18.658 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":1,"fsid":"680ee126-cfee-11ee-95bb-87774f69a715","modified":"2024-02-20T12:49:44.699059Z","created":"2024-02-20T12:49:44.699059Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi064","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:3300","nonce":0},{"type":"v1","addr":"172.21.15.64:6789","nonce":0}]},"addr":"172.21.15.64:6789/0","public_addr":"172.21.15.64:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-20T12:57:18.659 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:18.658+0000 7fec18d8a700 1 -- 172.21.15.71:0/1365005080 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fec00031090 msgr2=0x7fec00033550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:57:18.659 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:18.658+0000 7fec18d8a700 1 --2- 172.21.15.71:0/1365005080 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fec00031090 0x7fec00033550 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 rx=0x7fec040096a0 tx=0x7fec04006b40).stop 2024-02-20T12:57:18.659 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:18.658+0000 7fec18d8a700 1 -- 172.21.15.71:0/1365005080 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fec140f5480 msgr2=0x7fec1410b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:57:18.659 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:18.658+0000 7fec18d8a700 1 --2- 172.21.15.71:0/1365005080 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fec140f5480 0x7fec1410b910 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 rx=0x7febfc0089f0 tx=0x7febfc00b040).stop 2024-02-20T12:57:18.659 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:18.659+0000 7fec18d8a700 1 -- 172.21.15.71:0/1365005080 shutdown_connections 2024-02-20T12:57:18.659 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:18.659+0000 7fec18d8a700 1 --2- 172.21.15.71:0/1365005080 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fec00031090 0x7fec00033550 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:57:18.659 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:18.659+0000 7fec18d8a700 1 --2- 172.21.15.71:0/1365005080 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fec140f5480 0x7fec1410b910 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:57:18.659 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:18.659+0000 7fec18d8a700 1 -- 172.21.15.71:0/1365005080 >> 172.21.15.71:0/1365005080 conn(0x7fec140f0be0 msgr2=0x7fec140f16d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:57:18.660 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:18.659+0000 7fec18d8a700 1 -- 172.21.15.71:0/1365005080 shutdown_connections 2024-02-20T12:57:18.660 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:18.659+0000 7fec18d8a700 1 -- 172.21.15.71:0/1365005080 wait complete. 2024-02-20T12:57:18.660 INFO:teuthology.orchestra.run.smithi071.stderr:dumped monmap epoch 1 2024-02-20T12:57:19.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:57:18 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:57:17.763351+0000 mgr.smithi064.vriyor (mgr.14156) 200 : cluster [DBG] pgmap v186: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:57:19.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:57:18 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:57:18.657860+0000 mon.smithi064 (mon.0) 222 : audit [DBG] from='client.? 172.21.15.71:0/1365005080' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-20T12:57:20.378 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-20T12:57:20.378 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/daemon-base:latest-pacific shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 680ee126-cfee-11ee-95bb-87774f69a715 -- ceph mon dump -f json 2024-02-20T12:57:21.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:57:20 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:57:19.763633+0000 mgr.smithi064.vriyor (mgr.14156) 201 : cluster [DBG] pgmap v187: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:57:22.399 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:22.398+0000 7efeddf7b700 1 -- 172.21.15.71:0/798524885 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efed80f6590 msgr2=0x7efed80f69b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:57:22.399 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:22.398+0000 7efeddf7b700 1 --2- 172.21.15.71:0/798524885 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efed80f6590 0x7efed80f69b0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 rx=0x7efecc004760 tx=0x7efecc009ad0).stop 2024-02-20T12:57:22.399 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:22.398+0000 7efeddf7b700 1 -- 172.21.15.71:0/798524885 shutdown_connections 2024-02-20T12:57:22.399 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:22.398+0000 7efeddf7b700 1 --2- 172.21.15.71:0/798524885 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efed80f6590 0x7efed80f69b0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:57:22.399 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:22.398+0000 7efeddf7b700 1 -- 172.21.15.71:0/798524885 >> 172.21.15.71:0/798524885 conn(0x7efed80f1cd0 msgr2=0x7efed80f4110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:57:22.399 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:22.398+0000 7efeddf7b700 1 -- 172.21.15.71:0/798524885 shutdown_connections 2024-02-20T12:57:22.400 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:22.398+0000 7efeddf7b700 1 -- 172.21.15.71:0/798524885 wait complete. 2024-02-20T12:57:22.400 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:22.398+0000 7efeddf7b700 1 Processor -- start 2024-02-20T12:57:22.400 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:22.399+0000 7efeddf7b700 1 -- start start 2024-02-20T12:57:22.400 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:22.399+0000 7efeddf7b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efed80f6590 0x7efed81067c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:57:22.400 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:22.399+0000 7efeddf7b700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efed8106cb0 con 0x7efed80f6590 2024-02-20T12:57:22.402 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:22.399+0000 7efed77fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efed80f6590 0x7efed81067c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:57:22.402 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:22.399+0000 7efed77fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efed80f6590 0x7efed81067c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.71:58700/0 (socket says 172.21.15.71:58700) 2024-02-20T12:57:22.402 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:22.399+0000 7efed77fe700 1 -- 172.21.15.71:0/929221218 learned_addr learned my addr 172.21.15.71:0/929221218 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-02-20T12:57:22.402 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:22.399+0000 7efed77fe700 1 -- 172.21.15.71:0/929221218 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efecc009d70 con 0x7efed80f6590 2024-02-20T12:57:22.402 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:22.399+0000 7efed77fe700 1 --2- 172.21.15.71:0/929221218 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efed80f6590 0x7efed81067c0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 rx=0x7efecc004170 tx=0x7efecc009ad0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:57:22.402 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:22.400+0000 7efebffff700 1 -- 172.21.15.71:0/929221218 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7efecc00e040 con 0x7efed80f6590 2024-02-20T12:57:22.402 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:22.400+0000 7efeddf7b700 1 -- 172.21.15.71:0/929221218 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efed8106eb0 con 0x7efed80f6590 2024-02-20T12:57:22.403 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:22.400+0000 7efebffff700 1 -- 172.21.15.71:0/929221218 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(25 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7efecc008ef0 con 0x7efed80f6590 2024-02-20T12:57:22.403 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:22.400+0000 7efebffff700 1 -- 172.21.15.71:0/929221218 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7efecc014c90 con 0x7efed80f6590 2024-02-20T12:57:22.403 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:22.400+0000 7efeddf7b700 1 -- 172.21.15.71:0/929221218 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efed8107340 con 0x7efed80f6590 2024-02-20T12:57:22.403 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:22.400+0000 7efebffff700 1 -- 172.21.15.71:0/929221218 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 37151+0+0 (secure 0 0 0) 0x7efecc014420 con 0x7efed80f6590 2024-02-20T12:57:22.403 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:22.401+0000 7efebffff700 1 --2- 172.21.15.71:0/929221218 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7efec0031090 0x7efec0033550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:57:22.403 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:22.401+0000 7efebffff700 1 -- 172.21.15.71:0/929221218 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1065+0+0 (secure 0 0 0) 0x7efecc042240 con 0x7efed80f6590 2024-02-20T12:57:22.403 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:22.401+0000 7efeddf7b700 1 -- 172.21.15.71:0/929221218 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efed8045bb0 con 0x7efed80f6590 2024-02-20T12:57:22.403 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:22.401+0000 7efed6ffd700 1 --2- 172.21.15.71:0/929221218 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7efec0031090 0x7efec0033550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:57:22.403 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:22.401+0000 7efed6ffd700 1 --2- 172.21.15.71:0/929221218 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7efec0031090 0x7efec0033550 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 rx=0x7efec80096a0 tx=0x7efec8006b40).ready entity=mgr.14156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:57:22.406 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:22.406+0000 7efebffff700 1 -- 172.21.15.71:0/929221218 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+153237 (secure 0 0 0) 0x7efecc012d00 con 0x7efed80f6590 2024-02-20T12:57:22.614 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:22.614+0000 7efeddf7b700 1 -- 172.21.15.71:0/929221218 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7efed80fc170 con 0x7efed80f6590 2024-02-20T12:57:22.615 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:22.614+0000 7efebffff700 1 -- 172.21.15.71:0/929221218 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7efecc012b20 con 0x7efed80f6590 2024-02-20T12:57:22.615 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-20T12:57:22.615 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":1,"fsid":"680ee126-cfee-11ee-95bb-87774f69a715","modified":"2024-02-20T12:49:44.699059Z","created":"2024-02-20T12:49:44.699059Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi064","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:3300","nonce":0},{"type":"v1","addr":"172.21.15.64:6789","nonce":0}]},"addr":"172.21.15.64:6789/0","public_addr":"172.21.15.64:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-20T12:57:22.617 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:22.616+0000 7efeddf7b700 1 -- 172.21.15.71:0/929221218 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7efec0031090 msgr2=0x7efec0033550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:57:22.617 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:22.616+0000 7efeddf7b700 1 --2- 172.21.15.71:0/929221218 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7efec0031090 0x7efec0033550 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 rx=0x7efec80096a0 tx=0x7efec8006b40).stop 2024-02-20T12:57:22.617 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:22.616+0000 7efeddf7b700 1 -- 172.21.15.71:0/929221218 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efed80f6590 msgr2=0x7efed81067c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:57:22.617 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:22.616+0000 7efeddf7b700 1 --2- 172.21.15.71:0/929221218 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efed80f6590 0x7efed81067c0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 rx=0x7efecc004170 tx=0x7efecc009ad0).stop 2024-02-20T12:57:22.617 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:22.616+0000 7efeddf7b700 1 -- 172.21.15.71:0/929221218 shutdown_connections 2024-02-20T12:57:22.617 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:22.616+0000 7efeddf7b700 1 --2- 172.21.15.71:0/929221218 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7efec0031090 0x7efec0033550 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:57:22.617 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:22.616+0000 7efeddf7b700 1 --2- 172.21.15.71:0/929221218 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efed80f6590 0x7efed81067c0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:57:22.617 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:22.616+0000 7efeddf7b700 1 -- 172.21.15.71:0/929221218 >> 172.21.15.71:0/929221218 conn(0x7efed80f1cd0 msgr2=0x7efed80f27c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:57:22.617 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:22.616+0000 7efeddf7b700 1 -- 172.21.15.71:0/929221218 shutdown_connections 2024-02-20T12:57:22.617 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:22.616+0000 7efeddf7b700 1 -- 172.21.15.71:0/929221218 wait complete. 2024-02-20T12:57:22.618 INFO:teuthology.orchestra.run.smithi071.stderr:dumped monmap epoch 1 2024-02-20T12:57:23.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:57:22 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:57:21.763962+0000 mgr.smithi064.vriyor (mgr.14156) 202 : cluster [DBG] pgmap v188: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:57:23.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:57:22 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:57:22.615284+0000 mon.smithi064 (mon.0) 223 : audit [DBG] from='client.? 172.21.15.71:0/929221218' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-20T12:57:24.411 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-20T12:57:24.411 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/daemon-base:latest-pacific shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 680ee126-cfee-11ee-95bb-87774f69a715 -- ceph mon dump -f json 2024-02-20T12:57:25.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:57:24 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:57:23.764355+0000 mgr.smithi064.vriyor (mgr.14156) 203 : cluster [DBG] pgmap v189: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:57:26.437 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:26.427+0000 7f886139d700 1 -- 172.21.15.71:0/1914457367 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f885c0f5480 msgr2=0x7f885c0f58a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:57:26.437 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:26.427+0000 7f886139d700 1 --2- 172.21.15.71:0/1914457367 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f885c0f5480 0x7f885c0f58a0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 rx=0x7f8850009fb0 tx=0x7f8850009b30).stop 2024-02-20T12:57:26.437 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:26.427+0000 7f886139d700 1 -- 172.21.15.71:0/1914457367 shutdown_connections 2024-02-20T12:57:26.437 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:26.427+0000 7f886139d700 1 --2- 172.21.15.71:0/1914457367 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f885c0f5480 0x7f885c0f58a0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:57:26.437 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:26.427+0000 7f886139d700 1 -- 172.21.15.71:0/1914457367 >> 172.21.15.71:0/1914457367 conn(0x7f885c0f0be0 msgr2=0x7f885c0f3000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:57:26.438 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:26.427+0000 7f886139d700 1 -- 172.21.15.71:0/1914457367 shutdown_connections 2024-02-20T12:57:26.438 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:26.427+0000 7f886139d700 1 -- 172.21.15.71:0/1914457367 wait complete. 2024-02-20T12:57:26.438 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:26.428+0000 7f886139d700 1 Processor -- start 2024-02-20T12:57:26.438 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:26.428+0000 7f886139d700 1 -- start start 2024-02-20T12:57:26.438 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:26.428+0000 7f886139d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f885c0f5480 0x7f885c1094f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:57:26.438 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:26.428+0000 7f886139d700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f885c060320 con 0x7f885c0f5480 2024-02-20T12:57:26.438 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:26.428+0000 7f885affd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f885c0f5480 0x7f885c1094f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:57:26.438 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:26.428+0000 7f885affd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f885c0f5480 0x7f885c1094f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.71:58710/0 (socket says 172.21.15.71:58710) 2024-02-20T12:57:26.438 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:26.428+0000 7f885affd700 1 -- 172.21.15.71:0/3368235376 learned_addr learned my addr 172.21.15.71:0/3368235376 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-02-20T12:57:26.438 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:26.429+0000 7f885affd700 1 -- 172.21.15.71:0/3368235376 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8850005040 con 0x7f885c0f5480 2024-02-20T12:57:26.438 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:26.429+0000 7f885affd700 1 --2- 172.21.15.71:0/3368235376 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f885c0f5480 0x7f885c1094f0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 rx=0x7f88500089f0 tx=0x7f885000b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:57:26.438 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:26.429+0000 7f88437fe700 1 -- 172.21.15.71:0/3368235376 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8850009320 con 0x7f885c0f5480 2024-02-20T12:57:26.438 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:26.429+0000 7f886139d700 1 -- 172.21.15.71:0/3368235376 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f885c060520 con 0x7f885c0f5480 2024-02-20T12:57:26.438 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:26.429+0000 7f88437fe700 1 -- 172.21.15.71:0/3368235376 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(25 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f8850013ec0 con 0x7f885c0f5480 2024-02-20T12:57:26.438 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:26.429+0000 7f88437fe700 1 -- 172.21.15.71:0/3368235376 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8850014d30 con 0x7f885c0f5480 2024-02-20T12:57:26.439 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:26.429+0000 7f886139d700 1 -- 172.21.15.71:0/3368235376 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f885c109ba0 con 0x7f885c0f5480 2024-02-20T12:57:26.439 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:26.430+0000 7f88437fe700 1 -- 172.21.15.71:0/3368235376 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 37151+0+0 (secure 0 0 0) 0x7f88500263a0 con 0x7f885c0f5480 2024-02-20T12:57:26.439 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:26.430+0000 7f88437fe700 1 --2- 172.21.15.71:0/3368235376 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f8844031090 0x7f8844033550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:57:26.439 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:26.430+0000 7f88437fe700 1 -- 172.21.15.71:0/3368235376 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1065+0+0 (secure 0 0 0) 0x7f8850042420 con 0x7f885c0f5480 2024-02-20T12:57:26.439 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:26.430+0000 7f886139d700 1 -- 172.21.15.71:0/3368235376 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f885c0591e0 con 0x7f885c0f5480 2024-02-20T12:57:26.439 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:26.430+0000 7f885a7fc700 1 --2- 172.21.15.71:0/3368235376 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f8844031090 0x7f8844033550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:57:26.439 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:26.431+0000 7f885a7fc700 1 --2- 172.21.15.71:0/3368235376 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f8844031090 0x7f8844033550 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 rx=0x7f884c0096a0 tx=0x7f884c006b40).ready entity=mgr.14156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:57:26.439 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:26.435+0000 7f88437fe700 1 -- 172.21.15.71:0/3368235376 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+153237 (secure 0 0 0) 0x7f88500266d0 con 0x7f885c0f5480 2024-02-20T12:57:26.646 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:26.645+0000 7f886139d700 1 -- 172.21.15.71:0/3368235376 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f885c0faf70 con 0x7f885c0f5480 2024-02-20T12:57:26.646 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:26.646+0000 7f88437fe700 1 -- 172.21.15.71:0/3368235376 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7f885002b020 con 0x7f885c0f5480 2024-02-20T12:57:26.647 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-20T12:57:26.647 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":1,"fsid":"680ee126-cfee-11ee-95bb-87774f69a715","modified":"2024-02-20T12:49:44.699059Z","created":"2024-02-20T12:49:44.699059Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi064","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:3300","nonce":0},{"type":"v1","addr":"172.21.15.64:6789","nonce":0}]},"addr":"172.21.15.64:6789/0","public_addr":"172.21.15.64:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-20T12:57:26.648 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:26.648+0000 7f886139d700 1 -- 172.21.15.71:0/3368235376 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f8844031090 msgr2=0x7f8844033550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:57:26.649 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:26.648+0000 7f886139d700 1 --2- 172.21.15.71:0/3368235376 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f8844031090 0x7f8844033550 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 rx=0x7f884c0096a0 tx=0x7f884c006b40).stop 2024-02-20T12:57:26.649 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:26.648+0000 7f886139d700 1 -- 172.21.15.71:0/3368235376 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f885c0f5480 msgr2=0x7f885c1094f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:57:26.649 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:26.648+0000 7f886139d700 1 --2- 172.21.15.71:0/3368235376 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f885c0f5480 0x7f885c1094f0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 rx=0x7f88500089f0 tx=0x7f885000b040).stop 2024-02-20T12:57:26.649 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:26.648+0000 7f886139d700 1 -- 172.21.15.71:0/3368235376 shutdown_connections 2024-02-20T12:57:26.649 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:26.648+0000 7f886139d700 1 --2- 172.21.15.71:0/3368235376 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f8844031090 0x7f8844033550 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:57:26.649 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:26.648+0000 7f886139d700 1 --2- 172.21.15.71:0/3368235376 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f885c0f5480 0x7f885c1094f0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:57:26.649 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:26.648+0000 7f886139d700 1 -- 172.21.15.71:0/3368235376 >> 172.21.15.71:0/3368235376 conn(0x7f885c0f0be0 msgr2=0x7f885c0f16d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:57:26.649 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:26.648+0000 7f886139d700 1 -- 172.21.15.71:0/3368235376 shutdown_connections 2024-02-20T12:57:26.649 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:26.648+0000 7f886139d700 1 -- 172.21.15.71:0/3368235376 wait complete. 2024-02-20T12:57:26.649 INFO:teuthology.orchestra.run.smithi071.stderr:dumped monmap epoch 1 2024-02-20T12:57:27.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:57:26 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:57:25.764622+0000 mgr.smithi064.vriyor (mgr.14156) 204 : cluster [DBG] pgmap v190: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:57:27.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:57:26 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:57:26.646890+0000 mon.smithi064 (mon.0) 224 : audit [DBG] from='client.? 172.21.15.71:0/3368235376' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-20T12:57:28.461 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-20T12:57:28.462 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/daemon-base:latest-pacific shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 680ee126-cfee-11ee-95bb-87774f69a715 -- ceph mon dump -f json 2024-02-20T12:57:29.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:57:28 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:57:27.764963+0000 mgr.smithi064.vriyor (mgr.14156) 205 : cluster [DBG] pgmap v191: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:57:30.434 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:30.413+0000 7fc424d29700 1 -- 172.21.15.71:0/3679260854 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc4200f5480 msgr2=0x7fc4200f58a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:57:30.434 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:30.413+0000 7fc424d29700 1 --2- 172.21.15.71:0/3679260854 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc4200f5480 0x7fc4200f58a0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 rx=0x7fc410009fb0 tx=0x7fc410009b30).stop 2024-02-20T12:57:30.434 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:30.413+0000 7fc424d29700 1 -- 172.21.15.71:0/3679260854 shutdown_connections 2024-02-20T12:57:30.434 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:30.413+0000 7fc424d29700 1 --2- 172.21.15.71:0/3679260854 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc4200f5480 0x7fc4200f58a0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:57:30.434 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:30.413+0000 7fc424d29700 1 -- 172.21.15.71:0/3679260854 >> 172.21.15.71:0/3679260854 conn(0x7fc4200f0be0 msgr2=0x7fc4200f3000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:57:30.434 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:30.413+0000 7fc424d29700 1 -- 172.21.15.71:0/3679260854 shutdown_connections 2024-02-20T12:57:30.435 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:30.413+0000 7fc424d29700 1 -- 172.21.15.71:0/3679260854 wait complete. 2024-02-20T12:57:30.435 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:30.414+0000 7fc424d29700 1 Processor -- start 2024-02-20T12:57:30.435 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:30.414+0000 7fc424d29700 1 -- start start 2024-02-20T12:57:30.435 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:30.414+0000 7fc424d29700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc4200f5480 0x7fc4201094f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:57:30.435 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:30.414+0000 7fc424d29700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc420060320 con 0x7fc4200f5480 2024-02-20T12:57:30.435 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:30.415+0000 7fc41e59c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc4200f5480 0x7fc4201094f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:57:30.436 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:30.415+0000 7fc41e59c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc4200f5480 0x7fc4201094f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.71:52986/0 (socket says 172.21.15.71:52986) 2024-02-20T12:57:30.436 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:30.415+0000 7fc41e59c700 1 -- 172.21.15.71:0/2107541589 learned_addr learned my addr 172.21.15.71:0/2107541589 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-02-20T12:57:30.436 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:30.415+0000 7fc41e59c700 1 -- 172.21.15.71:0/2107541589 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc410005040 con 0x7fc4200f5480 2024-02-20T12:57:30.436 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:30.415+0000 7fc41e59c700 1 --2- 172.21.15.71:0/2107541589 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc4200f5480 0x7fc4201094f0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 rx=0x7fc4100089f0 tx=0x7fc41000b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:57:30.436 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:30.415+0000 7fc416ffd700 1 -- 172.21.15.71:0/2107541589 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc410009320 con 0x7fc4200f5480 2024-02-20T12:57:30.437 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:30.415+0000 7fc424d29700 1 -- 172.21.15.71:0/2107541589 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc420060520 con 0x7fc4200f5480 2024-02-20T12:57:30.437 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:30.415+0000 7fc416ffd700 1 -- 172.21.15.71:0/2107541589 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(25 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7fc410013ec0 con 0x7fc4200f5480 2024-02-20T12:57:30.437 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:30.415+0000 7fc416ffd700 1 -- 172.21.15.71:0/2107541589 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc410014d30 con 0x7fc4200f5480 2024-02-20T12:57:30.437 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:30.415+0000 7fc424d29700 1 -- 172.21.15.71:0/2107541589 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc420109ba0 con 0x7fc4200f5480 2024-02-20T12:57:30.438 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:30.416+0000 7fc416ffd700 1 -- 172.21.15.71:0/2107541589 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 37151+0+0 (secure 0 0 0) 0x7fc410014420 con 0x7fc4200f5480 2024-02-20T12:57:30.438 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:30.416+0000 7fc424d29700 1 -- 172.21.15.71:0/2107541589 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc420045bb0 con 0x7fc4200f5480 2024-02-20T12:57:30.438 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:30.416+0000 7fc416ffd700 1 --2- 172.21.15.71:0/2107541589 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fc40c031090 0x7fc40c033550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:57:30.438 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:30.416+0000 7fc416ffd700 1 -- 172.21.15.71:0/2107541589 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1065+0+0 (secure 0 0 0) 0x7fc410042480 con 0x7fc4200f5480 2024-02-20T12:57:30.439 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:30.416+0000 7fc41dd9b700 1 --2- 172.21.15.71:0/2107541589 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fc40c031090 0x7fc40c033550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:57:30.439 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:30.417+0000 7fc41dd9b700 1 --2- 172.21.15.71:0/2107541589 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fc40c031090 0x7fc40c033550 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 rx=0x7fc4080096a0 tx=0x7fc408006b40).ready entity=mgr.14156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:57:30.439 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:30.421+0000 7fc416ffd700 1 -- 172.21.15.71:0/2107541589 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+153237 (secure 0 0 0) 0x7fc41001d980 con 0x7fc4200f5480 2024-02-20T12:57:30.629 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:30.626+0000 7fc424d29700 1 -- 172.21.15.71:0/2107541589 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fc4200faf70 con 0x7fc4200f5480 2024-02-20T12:57:30.630 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:30.627+0000 7fc416ffd700 1 -- 172.21.15.71:0/2107541589 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7fc41001d380 con 0x7fc4200f5480 2024-02-20T12:57:30.630 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-20T12:57:30.630 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":1,"fsid":"680ee126-cfee-11ee-95bb-87774f69a715","modified":"2024-02-20T12:49:44.699059Z","created":"2024-02-20T12:49:44.699059Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi064","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:3300","nonce":0},{"type":"v1","addr":"172.21.15.64:6789","nonce":0}]},"addr":"172.21.15.64:6789/0","public_addr":"172.21.15.64:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-20T12:57:30.631 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:30.629+0000 7fc424d29700 1 -- 172.21.15.71:0/2107541589 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fc40c031090 msgr2=0x7fc40c033550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:57:30.631 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:30.629+0000 7fc424d29700 1 --2- 172.21.15.71:0/2107541589 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fc40c031090 0x7fc40c033550 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 rx=0x7fc4080096a0 tx=0x7fc408006b40).stop 2024-02-20T12:57:30.631 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:30.629+0000 7fc424d29700 1 -- 172.21.15.71:0/2107541589 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc4200f5480 msgr2=0x7fc4201094f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:57:30.631 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:30.629+0000 7fc424d29700 1 --2- 172.21.15.71:0/2107541589 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc4200f5480 0x7fc4201094f0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 rx=0x7fc4100089f0 tx=0x7fc41000b040).stop 2024-02-20T12:57:30.632 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:30.629+0000 7fc424d29700 1 -- 172.21.15.71:0/2107541589 shutdown_connections 2024-02-20T12:57:30.632 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:30.629+0000 7fc424d29700 1 --2- 172.21.15.71:0/2107541589 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fc40c031090 0x7fc40c033550 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:57:30.632 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:30.629+0000 7fc424d29700 1 --2- 172.21.15.71:0/2107541589 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc4200f5480 0x7fc4201094f0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:57:30.632 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:30.629+0000 7fc424d29700 1 -- 172.21.15.71:0/2107541589 >> 172.21.15.71:0/2107541589 conn(0x7fc4200f0be0 msgr2=0x7fc4200f16d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:57:30.632 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:30.629+0000 7fc424d29700 1 -- 172.21.15.71:0/2107541589 shutdown_connections 2024-02-20T12:57:30.632 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:30.629+0000 7fc424d29700 1 -- 172.21.15.71:0/2107541589 wait complete. 2024-02-20T12:57:30.632 INFO:teuthology.orchestra.run.smithi071.stderr:dumped monmap epoch 1 2024-02-20T12:57:31.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:57:30 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:57:29.765319+0000 mgr.smithi064.vriyor (mgr.14156) 206 : cluster [DBG] pgmap v192: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:57:31.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:57:30 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:57:30.628902+0000 mon.smithi064 (mon.0) 225 : audit [DBG] from='client.? 172.21.15.71:0/2107541589' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-20T12:57:32.476 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-20T12:57:32.477 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/daemon-base:latest-pacific shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 680ee126-cfee-11ee-95bb-87774f69a715 -- ceph mon dump -f json 2024-02-20T12:57:33.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:57:32 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:57:31.765657+0000 mgr.smithi064.vriyor (mgr.14156) 207 : cluster [DBG] pgmap v193: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:57:34.421 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:34.417+0000 7fc98e118700 1 -- 172.21.15.71:0/3267934630 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc9880f6590 msgr2=0x7fc9880f69b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:57:34.421 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:34.417+0000 7fc98e118700 1 --2- 172.21.15.71:0/3267934630 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc9880f6590 0x7fc9880f69b0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 rx=0x7fc970004760 tx=0x7fc970009ad0).stop 2024-02-20T12:57:34.421 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:34.417+0000 7fc98e118700 1 -- 172.21.15.71:0/3267934630 shutdown_connections 2024-02-20T12:57:34.421 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:34.417+0000 7fc98e118700 1 --2- 172.21.15.71:0/3267934630 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc9880f6590 0x7fc9880f69b0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:57:34.421 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:34.417+0000 7fc98e118700 1 -- 172.21.15.71:0/3267934630 >> 172.21.15.71:0/3267934630 conn(0x7fc9880f1cd0 msgr2=0x7fc9880f4110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:57:34.421 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:34.417+0000 7fc98e118700 1 -- 172.21.15.71:0/3267934630 shutdown_connections 2024-02-20T12:57:34.421 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:34.417+0000 7fc98e118700 1 -- 172.21.15.71:0/3267934630 wait complete. 2024-02-20T12:57:34.421 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:34.418+0000 7fc98e118700 1 Processor -- start 2024-02-20T12:57:34.421 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:34.418+0000 7fc98e118700 1 -- start start 2024-02-20T12:57:34.421 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:34.418+0000 7fc98e118700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc9880f6590 0x7fc9881067c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:57:34.422 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:34.418+0000 7fc98e118700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc988106cb0 con 0x7fc9880f6590 2024-02-20T12:57:34.422 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:34.419+0000 7fc9877fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc9880f6590 0x7fc9881067c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:57:34.422 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:34.419+0000 7fc9877fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc9880f6590 0x7fc9881067c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.71:53016/0 (socket says 172.21.15.71:53016) 2024-02-20T12:57:34.422 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:34.419+0000 7fc9877fe700 1 -- 172.21.15.71:0/3081600786 learned_addr learned my addr 172.21.15.71:0/3081600786 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-02-20T12:57:34.422 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:34.419+0000 7fc9877fe700 1 -- 172.21.15.71:0/3081600786 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc970009d70 con 0x7fc9880f6590 2024-02-20T12:57:34.422 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:34.419+0000 7fc9877fe700 1 --2- 172.21.15.71:0/3081600786 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc9880f6590 0x7fc9881067c0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 rx=0x7fc970004170 tx=0x7fc970009ad0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:57:34.422 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:34.419+0000 7fc97ffff700 1 -- 172.21.15.71:0/3081600786 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc97000e040 con 0x7fc9880f6590 2024-02-20T12:57:34.422 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:34.419+0000 7fc98e118700 1 -- 172.21.15.71:0/3081600786 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc988106eb0 con 0x7fc9880f6590 2024-02-20T12:57:34.422 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:34.419+0000 7fc97ffff700 1 -- 172.21.15.71:0/3081600786 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(25 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7fc970008ef0 con 0x7fc9880f6590 2024-02-20T12:57:34.422 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:34.419+0000 7fc97ffff700 1 -- 172.21.15.71:0/3081600786 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc970014c90 con 0x7fc9880f6590 2024-02-20T12:57:34.422 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:34.419+0000 7fc98e118700 1 -- 172.21.15.71:0/3081600786 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc988107340 con 0x7fc9880f6590 2024-02-20T12:57:34.423 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:34.420+0000 7fc97ffff700 1 -- 172.21.15.71:0/3081600786 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 37151+0+0 (secure 0 0 0) 0x7fc970014420 con 0x7fc9880f6590 2024-02-20T12:57:34.423 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:34.420+0000 7fc97ffff700 1 --2- 172.21.15.71:0/3081600786 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fc974031090 0x7fc974033550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:57:34.423 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:34.420+0000 7fc98e118700 1 -- 172.21.15.71:0/3081600786 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc988045bb0 con 0x7fc9880f6590 2024-02-20T12:57:34.423 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:34.420+0000 7fc97ffff700 1 -- 172.21.15.71:0/3081600786 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1065+0+0 (secure 0 0 0) 0x7fc9700422b0 con 0x7fc9880f6590 2024-02-20T12:57:34.423 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:34.420+0000 7fc986ffd700 1 --2- 172.21.15.71:0/3081600786 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fc974031090 0x7fc974033550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:57:34.423 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:34.421+0000 7fc986ffd700 1 --2- 172.21.15.71:0/3081600786 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fc974031090 0x7fc974033550 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 rx=0x7fc9780096a0 tx=0x7fc978006b40).ready entity=mgr.14156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:57:34.426 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:34.425+0000 7fc97ffff700 1 -- 172.21.15.71:0/3081600786 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+153237 (secure 0 0 0) 0x7fc970026300 con 0x7fc9880f6590 2024-02-20T12:57:34.635 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:34.634+0000 7fc98e118700 1 -- 172.21.15.71:0/3081600786 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fc9881081c0 con 0x7fc9880f6590 2024-02-20T12:57:34.635 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:34.634+0000 7fc97ffff700 1 -- 172.21.15.71:0/3081600786 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7fc970012b20 con 0x7fc9880f6590 2024-02-20T12:57:34.636 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-20T12:57:34.636 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":1,"fsid":"680ee126-cfee-11ee-95bb-87774f69a715","modified":"2024-02-20T12:49:44.699059Z","created":"2024-02-20T12:49:44.699059Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi064","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:3300","nonce":0},{"type":"v1","addr":"172.21.15.64:6789","nonce":0}]},"addr":"172.21.15.64:6789/0","public_addr":"172.21.15.64:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-20T12:57:34.637 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:34.636+0000 7fc98e118700 1 -- 172.21.15.71:0/3081600786 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fc974031090 msgr2=0x7fc974033550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:57:34.637 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:34.636+0000 7fc98e118700 1 --2- 172.21.15.71:0/3081600786 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fc974031090 0x7fc974033550 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 rx=0x7fc9780096a0 tx=0x7fc978006b40).stop 2024-02-20T12:57:34.637 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:34.636+0000 7fc98e118700 1 -- 172.21.15.71:0/3081600786 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc9880f6590 msgr2=0x7fc9881067c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:57:34.637 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:34.636+0000 7fc98e118700 1 --2- 172.21.15.71:0/3081600786 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc9880f6590 0x7fc9881067c0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 rx=0x7fc970004170 tx=0x7fc970009ad0).stop 2024-02-20T12:57:34.638 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:34.636+0000 7fc98e118700 1 -- 172.21.15.71:0/3081600786 shutdown_connections 2024-02-20T12:57:34.638 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:34.636+0000 7fc98e118700 1 --2- 172.21.15.71:0/3081600786 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fc974031090 0x7fc974033550 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:57:34.638 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:34.636+0000 7fc98e118700 1 --2- 172.21.15.71:0/3081600786 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc9880f6590 0x7fc9881067c0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:57:34.638 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:34.636+0000 7fc98e118700 1 -- 172.21.15.71:0/3081600786 >> 172.21.15.71:0/3081600786 conn(0x7fc9880f1cd0 msgr2=0x7fc9880f27c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:57:34.638 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:34.636+0000 7fc98e118700 1 -- 172.21.15.71:0/3081600786 shutdown_connections 2024-02-20T12:57:34.638 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:34.636+0000 7fc98e118700 1 -- 172.21.15.71:0/3081600786 wait complete. 2024-02-20T12:57:34.638 INFO:teuthology.orchestra.run.smithi071.stderr:dumped monmap epoch 1 2024-02-20T12:57:35.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:57:34 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:57:33.766026+0000 mgr.smithi064.vriyor (mgr.14156) 208 : cluster [DBG] pgmap v194: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:57:35.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:57:34 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:57:34.636116+0000 mon.smithi064 (mon.0) 226 : audit [DBG] from='client.? 172.21.15.71:0/3081600786' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-20T12:57:36.283 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-20T12:57:36.283 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/daemon-base:latest-pacific shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 680ee126-cfee-11ee-95bb-87774f69a715 -- ceph mon dump -f json 2024-02-20T12:57:37.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:57:36 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:57:35.766330+0000 mgr.smithi064.vriyor (mgr.14156) 209 : cluster [DBG] pgmap v195: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:57:38.294 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:38.291+0000 7fc80ffaf700 1 -- 172.21.15.71:0/892814610 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc8080f6590 msgr2=0x7fc8080f69b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:57:38.294 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:38.291+0000 7fc80ffaf700 1 --2- 172.21.15.71:0/892814610 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc8080f6590 0x7fc8080f69b0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 rx=0x7fc804004760 tx=0x7fc804009ad0).stop 2024-02-20T12:57:38.294 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:38.291+0000 7fc80ffaf700 1 -- 172.21.15.71:0/892814610 shutdown_connections 2024-02-20T12:57:38.294 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:38.291+0000 7fc80ffaf700 1 --2- 172.21.15.71:0/892814610 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc8080f6590 0x7fc8080f69b0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:57:38.294 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:38.291+0000 7fc80ffaf700 1 -- 172.21.15.71:0/892814610 >> 172.21.15.71:0/892814610 conn(0x7fc8080f1cd0 msgr2=0x7fc8080f4110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:57:38.294 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:38.291+0000 7fc80ffaf700 1 -- 172.21.15.71:0/892814610 shutdown_connections 2024-02-20T12:57:38.294 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:38.291+0000 7fc80ffaf700 1 -- 172.21.15.71:0/892814610 wait complete. 2024-02-20T12:57:38.294 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:38.292+0000 7fc80ffaf700 1 Processor -- start 2024-02-20T12:57:38.294 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:38.292+0000 7fc80ffaf700 1 -- start start 2024-02-20T12:57:38.294 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:38.292+0000 7fc80ffaf700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc8080f6590 0x7fc8081067c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:57:38.294 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:38.292+0000 7fc80ffaf700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc808106cb0 con 0x7fc8080f6590 2024-02-20T12:57:38.295 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:38.292+0000 7fc80dd4b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc8080f6590 0x7fc8081067c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:57:38.295 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:38.292+0000 7fc80dd4b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc8080f6590 0x7fc8081067c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.71:54106/0 (socket says 172.21.15.71:54106) 2024-02-20T12:57:38.295 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:38.292+0000 7fc80dd4b700 1 -- 172.21.15.71:0/1155930942 learned_addr learned my addr 172.21.15.71:0/1155930942 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-02-20T12:57:38.296 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:38.293+0000 7fc80dd4b700 1 -- 172.21.15.71:0/1155930942 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc804009d70 con 0x7fc8080f6590 2024-02-20T12:57:38.297 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:38.293+0000 7fc80dd4b700 1 --2- 172.21.15.71:0/1155930942 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc8080f6590 0x7fc8081067c0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 rx=0x7fc804004170 tx=0x7fc804009ad0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:57:38.297 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:38.293+0000 7fc7fe7fc700 1 -- 172.21.15.71:0/1155930942 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc80400e040 con 0x7fc8080f6590 2024-02-20T12:57:38.297 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:38.293+0000 7fc80ffaf700 1 -- 172.21.15.71:0/1155930942 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc808106eb0 con 0x7fc8080f6590 2024-02-20T12:57:38.297 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:38.293+0000 7fc7fe7fc700 1 -- 172.21.15.71:0/1155930942 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(25 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7fc804008ef0 con 0x7fc8080f6590 2024-02-20T12:57:38.297 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:38.293+0000 7fc7fe7fc700 1 -- 172.21.15.71:0/1155930942 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc804014c90 con 0x7fc8080f6590 2024-02-20T12:57:38.297 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:38.293+0000 7fc80ffaf700 1 -- 172.21.15.71:0/1155930942 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc808107340 con 0x7fc8080f6590 2024-02-20T12:57:38.297 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:38.294+0000 7fc7fe7fc700 1 -- 172.21.15.71:0/1155930942 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 37151+0+0 (secure 0 0 0) 0x7fc804014420 con 0x7fc8080f6590 2024-02-20T12:57:38.297 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:38.294+0000 7fc80ffaf700 1 -- 172.21.15.71:0/1155930942 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc808045bb0 con 0x7fc8080f6590 2024-02-20T12:57:38.297 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:38.294+0000 7fc7fe7fc700 1 --2- 172.21.15.71:0/1155930942 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fc7f4031090 0x7fc7f4033550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:57:38.297 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:38.294+0000 7fc7fe7fc700 1 -- 172.21.15.71:0/1155930942 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1065+0+0 (secure 0 0 0) 0x7fc8040422b0 con 0x7fc8080f6590 2024-02-20T12:57:38.297 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:38.294+0000 7fc80d54a700 1 --2- 172.21.15.71:0/1155930942 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fc7f4031090 0x7fc7f4033550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:57:38.297 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:38.295+0000 7fc80d54a700 1 --2- 172.21.15.71:0/1155930942 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fc7f4031090 0x7fc7f4033550 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 rx=0x7fc7f80096a0 tx=0x7fc7f8006b40).ready entity=mgr.14156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:57:38.300 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:38.299+0000 7fc7fe7fc700 1 -- 172.21.15.71:0/1155930942 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+153237 (secure 0 0 0) 0x7fc804026300 con 0x7fc8080f6590 2024-02-20T12:57:38.509 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:38.508+0000 7fc80ffaf700 1 -- 172.21.15.71:0/1155930942 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fc8081081f0 con 0x7fc8080f6590 2024-02-20T12:57:38.510 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:38.508+0000 7fc7fe7fc700 1 -- 172.21.15.71:0/1155930942 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7fc804012b20 con 0x7fc8080f6590 2024-02-20T12:57:38.510 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-20T12:57:38.510 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":1,"fsid":"680ee126-cfee-11ee-95bb-87774f69a715","modified":"2024-02-20T12:49:44.699059Z","created":"2024-02-20T12:49:44.699059Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi064","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:3300","nonce":0},{"type":"v1","addr":"172.21.15.64:6789","nonce":0}]},"addr":"172.21.15.64:6789/0","public_addr":"172.21.15.64:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-20T12:57:38.511 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:38.510+0000 7fc80ffaf700 1 -- 172.21.15.71:0/1155930942 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fc7f4031090 msgr2=0x7fc7f4033550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:57:38.511 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:38.510+0000 7fc80ffaf700 1 --2- 172.21.15.71:0/1155930942 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fc7f4031090 0x7fc7f4033550 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 rx=0x7fc7f80096a0 tx=0x7fc7f8006b40).stop 2024-02-20T12:57:38.512 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:38.510+0000 7fc80ffaf700 1 -- 172.21.15.71:0/1155930942 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc8080f6590 msgr2=0x7fc8081067c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:57:38.512 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:38.510+0000 7fc80ffaf700 1 --2- 172.21.15.71:0/1155930942 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc8080f6590 0x7fc8081067c0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 rx=0x7fc804004170 tx=0x7fc804009ad0).stop 2024-02-20T12:57:38.512 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:38.510+0000 7fc80ffaf700 1 -- 172.21.15.71:0/1155930942 shutdown_connections 2024-02-20T12:57:38.512 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:38.510+0000 7fc80ffaf700 1 --2- 172.21.15.71:0/1155930942 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fc7f4031090 0x7fc7f4033550 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:57:38.512 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:38.510+0000 7fc80ffaf700 1 --2- 172.21.15.71:0/1155930942 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc8080f6590 0x7fc8081067c0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:57:38.512 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:38.510+0000 7fc80ffaf700 1 -- 172.21.15.71:0/1155930942 >> 172.21.15.71:0/1155930942 conn(0x7fc8080f1cd0 msgr2=0x7fc8080f27c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:57:38.512 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:38.510+0000 7fc80ffaf700 1 -- 172.21.15.71:0/1155930942 shutdown_connections 2024-02-20T12:57:38.512 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:38.510+0000 7fc80ffaf700 1 -- 172.21.15.71:0/1155930942 wait complete. 2024-02-20T12:57:38.512 INFO:teuthology.orchestra.run.smithi071.stderr:dumped monmap epoch 1 2024-02-20T12:57:39.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:57:38 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:57:37.766612+0000 mgr.smithi064.vriyor (mgr.14156) 210 : cluster [DBG] pgmap v196: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:57:39.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:57:38 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:57:38.510145+0000 mon.smithi064 (mon.0) 227 : audit [DBG] from='client.? 172.21.15.71:0/1155930942' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-20T12:57:40.293 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-20T12:57:40.294 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/daemon-base:latest-pacific shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 680ee126-cfee-11ee-95bb-87774f69a715 -- ceph mon dump -f json 2024-02-20T12:57:41.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:57:40 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:57:39.766984+0000 mgr.smithi064.vriyor (mgr.14156) 211 : cluster [DBG] pgmap v197: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:57:42.251 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:42.249+0000 7f8b3e72b700 1 -- 172.21.15.71:0/3816878330 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8b380f5480 msgr2=0x7f8b380f58a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:57:42.251 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:42.249+0000 7f8b3e72b700 1 --2- 172.21.15.71:0/3816878330 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8b380f5480 0x7f8b380f58a0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 rx=0x7f8b28009fb0 tx=0x7f8b28009b30).stop 2024-02-20T12:57:42.251 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:42.249+0000 7f8b3e72b700 1 -- 172.21.15.71:0/3816878330 shutdown_connections 2024-02-20T12:57:42.251 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:42.249+0000 7f8b3e72b700 1 --2- 172.21.15.71:0/3816878330 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8b380f5480 0x7f8b380f58a0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:57:42.251 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:42.249+0000 7f8b3e72b700 1 -- 172.21.15.71:0/3816878330 >> 172.21.15.71:0/3816878330 conn(0x7f8b380f0be0 msgr2=0x7f8b380f3000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:57:42.251 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:42.249+0000 7f8b3e72b700 1 -- 172.21.15.71:0/3816878330 shutdown_connections 2024-02-20T12:57:42.252 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:42.249+0000 7f8b3e72b700 1 -- 172.21.15.71:0/3816878330 wait complete. 2024-02-20T12:57:42.252 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:42.249+0000 7f8b3e72b700 1 Processor -- start 2024-02-20T12:57:42.252 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:42.250+0000 7f8b3e72b700 1 -- start start 2024-02-20T12:57:42.252 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:42.250+0000 7f8b3e72b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8b380f5480 0x7f8b380fd460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:57:42.252 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:42.250+0000 7f8b3e72b700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b380fb8e0 con 0x7f8b380f5480 2024-02-20T12:57:42.254 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:42.250+0000 7f8b37fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8b380f5480 0x7f8b380fd460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:57:42.254 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:42.250+0000 7f8b37fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8b380f5480 0x7f8b380fd460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.71:54122/0 (socket says 172.21.15.71:54122) 2024-02-20T12:57:42.254 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:42.250+0000 7f8b37fff700 1 -- 172.21.15.71:0/3895905281 learned_addr learned my addr 172.21.15.71:0/3895905281 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-02-20T12:57:42.254 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:42.250+0000 7f8b37fff700 1 -- 172.21.15.71:0/3895905281 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b28005040 con 0x7f8b380f5480 2024-02-20T12:57:42.254 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:42.250+0000 7f8b37fff700 1 --2- 172.21.15.71:0/3895905281 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8b380f5480 0x7f8b380fd460 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 rx=0x7f8b28000c00 tx=0x7f8b2800b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:57:42.254 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:42.251+0000 7f8b34ff9700 1 -- 172.21.15.71:0/3895905281 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8b280095f0 con 0x7f8b380f5480 2024-02-20T12:57:42.254 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:42.251+0000 7f8b3e72b700 1 -- 172.21.15.71:0/3895905281 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b380fba80 con 0x7f8b380f5480 2024-02-20T12:57:42.254 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:42.251+0000 7f8b34ff9700 1 -- 172.21.15.71:0/3895905281 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(25 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f8b28007e50 con 0x7f8b380f5480 2024-02-20T12:57:42.254 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:42.251+0000 7f8b34ff9700 1 -- 172.21.15.71:0/3895905281 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8b28014e70 con 0x7f8b380f5480 2024-02-20T12:57:42.254 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:42.251+0000 7f8b3e72b700 1 -- 172.21.15.71:0/3895905281 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b380fbf10 con 0x7f8b380f5480 2024-02-20T12:57:42.254 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:42.251+0000 7f8b34ff9700 1 -- 172.21.15.71:0/3895905281 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 37151+0+0 (secure 0 0 0) 0x7f8b28025070 con 0x7f8b380f5480 2024-02-20T12:57:42.254 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:42.252+0000 7f8b34ff9700 1 --2- 172.21.15.71:0/3895905281 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f8b20031090 0x7f8b20033550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:57:42.255 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:42.252+0000 7f8b34ff9700 1 -- 172.21.15.71:0/3895905281 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1065+0+0 (secure 0 0 0) 0x7f8b28041d10 con 0x7f8b380f5480 2024-02-20T12:57:42.255 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:42.252+0000 7f8b3e72b700 1 -- 172.21.15.71:0/3895905281 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b380591e0 con 0x7f8b380f5480 2024-02-20T12:57:42.255 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:42.252+0000 7f8b377fe700 1 --2- 172.21.15.71:0/3895905281 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f8b20031090 0x7f8b20033550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:57:42.255 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:42.252+0000 7f8b377fe700 1 --2- 172.21.15.71:0/3895905281 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f8b20031090 0x7f8b20033550 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 rx=0x7f8b2c0096a0 tx=0x7f8b2c006b40).ready entity=mgr.14156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:57:42.258 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:42.257+0000 7f8b34ff9700 1 -- 172.21.15.71:0/3895905281 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+153237 (secure 0 0 0) 0x7f8b280079c0 con 0x7f8b380f5480 2024-02-20T12:57:42.464 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:42.463+0000 7f8b3e72b700 1 -- 172.21.15.71:0/3895905281 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f8b38045bb0 con 0x7f8b380f5480 2024-02-20T12:57:42.465 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:42.463+0000 7f8b34ff9700 1 -- 172.21.15.71:0/3895905281 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7f8b2802d090 con 0x7f8b380f5480 2024-02-20T12:57:42.465 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-20T12:57:42.465 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":1,"fsid":"680ee126-cfee-11ee-95bb-87774f69a715","modified":"2024-02-20T12:49:44.699059Z","created":"2024-02-20T12:49:44.699059Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi064","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:3300","nonce":0},{"type":"v1","addr":"172.21.15.64:6789","nonce":0}]},"addr":"172.21.15.64:6789/0","public_addr":"172.21.15.64:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-20T12:57:42.466 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:42.465+0000 7f8b3e72b700 1 -- 172.21.15.71:0/3895905281 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f8b20031090 msgr2=0x7f8b20033550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:57:42.467 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:42.465+0000 7f8b3e72b700 1 --2- 172.21.15.71:0/3895905281 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f8b20031090 0x7f8b20033550 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 rx=0x7f8b2c0096a0 tx=0x7f8b2c006b40).stop 2024-02-20T12:57:42.467 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:42.465+0000 7f8b3e72b700 1 -- 172.21.15.71:0/3895905281 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8b380f5480 msgr2=0x7f8b380fd460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:57:42.467 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:42.465+0000 7f8b3e72b700 1 --2- 172.21.15.71:0/3895905281 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8b380f5480 0x7f8b380fd460 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 rx=0x7f8b28000c00 tx=0x7f8b2800b040).stop 2024-02-20T12:57:42.467 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:42.465+0000 7f8b3e72b700 1 -- 172.21.15.71:0/3895905281 shutdown_connections 2024-02-20T12:57:42.467 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:42.465+0000 7f8b3e72b700 1 --2- 172.21.15.71:0/3895905281 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f8b20031090 0x7f8b20033550 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:57:42.467 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:42.465+0000 7f8b3e72b700 1 --2- 172.21.15.71:0/3895905281 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8b380f5480 0x7f8b380fd460 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:57:42.467 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:42.465+0000 7f8b3e72b700 1 -- 172.21.15.71:0/3895905281 >> 172.21.15.71:0/3895905281 conn(0x7f8b380f0be0 msgr2=0x7f8b3805fee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:57:42.467 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:42.465+0000 7f8b3e72b700 1 -- 172.21.15.71:0/3895905281 shutdown_connections 2024-02-20T12:57:42.467 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:42.465+0000 7f8b3e72b700 1 -- 172.21.15.71:0/3895905281 wait complete. 2024-02-20T12:57:42.467 INFO:teuthology.orchestra.run.smithi071.stderr:dumped monmap epoch 1 2024-02-20T12:57:43.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:57:42 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:57:41.767257+0000 mgr.smithi064.vriyor (mgr.14156) 212 : cluster [DBG] pgmap v198: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:57:43.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:57:42 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:57:42.465324+0000 mon.smithi064 (mon.0) 228 : audit [DBG] from='client.? 172.21.15.71:0/3895905281' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-20T12:57:44.188 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-20T12:57:44.213 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/daemon-base:latest-pacific shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 680ee126-cfee-11ee-95bb-87774f69a715 -- ceph mon dump -f json 2024-02-20T12:57:45.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:57:44 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:57:43.767569+0000 mgr.smithi064.vriyor (mgr.14156) 213 : cluster [DBG] pgmap v199: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:57:46.184 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:46.166+0000 7f556f0f8700 1 -- 172.21.15.71:0/1203591170 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f55680f52d0 msgr2=0x7f55680f56f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:57:46.184 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:46.166+0000 7f556f0f8700 1 --2- 172.21.15.71:0/1203591170 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f55680f52d0 0x7f55680f56f0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 rx=0x7f5558009fb0 tx=0x7f5558009b30).stop 2024-02-20T12:57:46.184 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:46.167+0000 7f556f0f8700 1 -- 172.21.15.71:0/1203591170 shutdown_connections 2024-02-20T12:57:46.184 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:46.167+0000 7f556f0f8700 1 --2- 172.21.15.71:0/1203591170 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f55680f52d0 0x7f55680f56f0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:57:46.184 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:46.167+0000 7f556f0f8700 1 -- 172.21.15.71:0/1203591170 >> 172.21.15.71:0/1203591170 conn(0x7f55680f0a30 msgr2=0x7f55680f2e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:57:46.185 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:46.167+0000 7f556f0f8700 1 -- 172.21.15.71:0/1203591170 shutdown_connections 2024-02-20T12:57:46.185 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:46.167+0000 7f556f0f8700 1 -- 172.21.15.71:0/1203591170 wait complete. 2024-02-20T12:57:46.185 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:46.167+0000 7f556f0f8700 1 Processor -- start 2024-02-20T12:57:46.185 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:46.167+0000 7f556f0f8700 1 -- start start 2024-02-20T12:57:46.185 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:46.167+0000 7f556f0f8700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f55680f52d0 0x7f5568106640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:57:46.185 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:46.167+0000 7f556f0f8700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5568106b30 con 0x7f55680f52d0 2024-02-20T12:57:46.185 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:46.167+0000 7f556e0f6700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f55680f52d0 0x7f5568106640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:57:46.185 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:46.167+0000 7f556e0f6700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f55680f52d0 0x7f5568106640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.71:54140/0 (socket says 172.21.15.71:54140) 2024-02-20T12:57:46.185 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:46.167+0000 7f556e0f6700 1 -- 172.21.15.71:0/2240559220 learned_addr learned my addr 172.21.15.71:0/2240559220 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-02-20T12:57:46.185 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:46.168+0000 7f556e0f6700 1 -- 172.21.15.71:0/2240559220 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5558005040 con 0x7f55680f52d0 2024-02-20T12:57:46.185 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:46.168+0000 7f556e0f6700 1 --2- 172.21.15.71:0/2240559220 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f55680f52d0 0x7f5568106640 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 rx=0x7f5558000c00 tx=0x7f555800b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:57:46.185 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:46.168+0000 7f555effd700 1 -- 172.21.15.71:0/2240559220 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f55580095f0 con 0x7f55680f52d0 2024-02-20T12:57:46.185 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:46.168+0000 7f555effd700 1 -- 172.21.15.71:0/2240559220 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(25 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f5558007e50 con 0x7f55680f52d0 2024-02-20T12:57:46.185 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:46.168+0000 7f556f0f8700 1 -- 172.21.15.71:0/2240559220 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5568106d30 con 0x7f55680f52d0 2024-02-20T12:57:46.186 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:46.168+0000 7f555effd700 1 -- 172.21.15.71:0/2240559220 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5558014e70 con 0x7f55680f52d0 2024-02-20T12:57:46.186 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:46.168+0000 7f556f0f8700 1 -- 172.21.15.71:0/2240559220 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55681071c0 con 0x7f55680f52d0 2024-02-20T12:57:46.186 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:46.169+0000 7f555effd700 1 -- 172.21.15.71:0/2240559220 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 37151+0+0 (secure 0 0 0) 0x7f5558025070 con 0x7f55680f52d0 2024-02-20T12:57:46.186 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:46.169+0000 7f555effd700 1 --2- 172.21.15.71:0/2240559220 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f5554031090 0x7f5554033550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:57:46.186 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:46.169+0000 7f555effd700 1 -- 172.21.15.71:0/2240559220 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1065+0+0 (secure 0 0 0) 0x7f5558041dd0 con 0x7f55680f52d0 2024-02-20T12:57:46.186 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:46.169+0000 7f556f0f8700 1 -- 172.21.15.71:0/2240559220 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5568045bb0 con 0x7f55680f52d0 2024-02-20T12:57:46.186 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:46.169+0000 7f556d8f5700 1 --2- 172.21.15.71:0/2240559220 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f5554031090 0x7f5554033550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:57:46.186 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:46.170+0000 7f556d8f5700 1 --2- 172.21.15.71:0/2240559220 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f5554031090 0x7f5554033550 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 rx=0x7f55640096a0 tx=0x7f5564006b40).ready entity=mgr.14156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:57:46.186 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:46.174+0000 7f555effd700 1 -- 172.21.15.71:0/2240559220 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+153237 (secure 0 0 0) 0x7f5558026350 con 0x7f55680f52d0 2024-02-20T12:57:46.382 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:46.380+0000 7f556f0f8700 1 -- 172.21.15.71:0/2240559220 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f5568107ee0 con 0x7f55680f52d0 2024-02-20T12:57:46.382 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:46.381+0000 7f555effd700 1 -- 172.21.15.71:0/2240559220 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7f5558012330 con 0x7f55680f52d0 2024-02-20T12:57:46.383 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-20T12:57:46.383 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":1,"fsid":"680ee126-cfee-11ee-95bb-87774f69a715","modified":"2024-02-20T12:49:44.699059Z","created":"2024-02-20T12:49:44.699059Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi064","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:3300","nonce":0},{"type":"v1","addr":"172.21.15.64:6789","nonce":0}]},"addr":"172.21.15.64:6789/0","public_addr":"172.21.15.64:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-20T12:57:46.384 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:46.382+0000 7f556f0f8700 1 -- 172.21.15.71:0/2240559220 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f5554031090 msgr2=0x7f5554033550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:57:46.384 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:46.382+0000 7f556f0f8700 1 --2- 172.21.15.71:0/2240559220 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f5554031090 0x7f5554033550 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 rx=0x7f55640096a0 tx=0x7f5564006b40).stop 2024-02-20T12:57:46.384 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:46.382+0000 7f556f0f8700 1 -- 172.21.15.71:0/2240559220 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f55680f52d0 msgr2=0x7f5568106640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:57:46.385 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:46.382+0000 7f556f0f8700 1 --2- 172.21.15.71:0/2240559220 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f55680f52d0 0x7f5568106640 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 rx=0x7f5558000c00 tx=0x7f555800b040).stop 2024-02-20T12:57:46.385 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:46.383+0000 7f556f0f8700 1 -- 172.21.15.71:0/2240559220 shutdown_connections 2024-02-20T12:57:46.385 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:46.383+0000 7f556f0f8700 1 --2- 172.21.15.71:0/2240559220 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f5554031090 0x7f5554033550 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:57:46.385 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:46.383+0000 7f556f0f8700 1 --2- 172.21.15.71:0/2240559220 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f55680f52d0 0x7f5568106640 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:57:46.385 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:46.383+0000 7f556f0f8700 1 -- 172.21.15.71:0/2240559220 >> 172.21.15.71:0/2240559220 conn(0x7f55680f0a30 msgr2=0x7f55680f1550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:57:46.385 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:46.383+0000 7f556f0f8700 1 -- 172.21.15.71:0/2240559220 shutdown_connections 2024-02-20T12:57:46.385 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:46.383+0000 7f556f0f8700 1 -- 172.21.15.71:0/2240559220 wait complete. 2024-02-20T12:57:46.385 INFO:teuthology.orchestra.run.smithi071.stderr:dumped monmap epoch 1 2024-02-20T12:57:47.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:57:46 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:57:45.767820+0000 mgr.smithi064.vriyor (mgr.14156) 214 : cluster [DBG] pgmap v200: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:57:47.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:57:46 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:57:46.382905+0000 mon.smithi064 (mon.0) 229 : audit [DBG] from='client.? 172.21.15.71:0/2240559220' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-20T12:57:47.975 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-20T12:57:47.975 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/daemon-base:latest-pacific shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 680ee126-cfee-11ee-95bb-87774f69a715 -- ceph mon dump -f json 2024-02-20T12:57:49.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:57:49 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:57:47.768219+0000 mgr.smithi064.vriyor (mgr.14156) 215 : cluster [DBG] pgmap v201: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:57:49.873 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:49.847+0000 7fa1831a0700 1 -- 172.21.15.71:0/841049542 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa17c0f52d0 msgr2=0x7fa17c0f56f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:57:49.873 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:49.847+0000 7fa1831a0700 1 --2- 172.21.15.71:0/841049542 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa17c0f52d0 0x7fa17c0f56f0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 rx=0x7fa16c009fb0 tx=0x7fa16c009b30).stop 2024-02-20T12:57:49.873 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:49.847+0000 7fa1831a0700 1 -- 172.21.15.71:0/841049542 shutdown_connections 2024-02-20T12:57:49.873 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:49.847+0000 7fa1831a0700 1 --2- 172.21.15.71:0/841049542 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa17c0f52d0 0x7fa17c0f56f0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:57:49.873 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:49.847+0000 7fa1831a0700 1 -- 172.21.15.71:0/841049542 >> 172.21.15.71:0/841049542 conn(0x7fa17c0f0a30 msgr2=0x7fa17c0f2e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:57:49.873 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:49.847+0000 7fa1831a0700 1 -- 172.21.15.71:0/841049542 shutdown_connections 2024-02-20T12:57:49.874 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:49.847+0000 7fa1831a0700 1 -- 172.21.15.71:0/841049542 wait complete. 2024-02-20T12:57:49.874 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:49.848+0000 7fa1831a0700 1 Processor -- start 2024-02-20T12:57:49.874 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:49.848+0000 7fa1831a0700 1 -- start start 2024-02-20T12:57:49.874 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:49.848+0000 7fa1831a0700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa17c0f52d0 0x7fa17c0fd2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:57:49.874 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:49.848+0000 7fa1831a0700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa17c0fb730 con 0x7fa17c0f52d0 2024-02-20T12:57:49.874 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:49.848+0000 7fa18219e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa17c0f52d0 0x7fa17c0fd2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:57:49.874 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:49.848+0000 7fa18219e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa17c0f52d0 0x7fa17c0fd2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.71:41664/0 (socket says 172.21.15.71:41664) 2024-02-20T12:57:49.874 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:49.848+0000 7fa18219e700 1 -- 172.21.15.71:0/3931645924 learned_addr learned my addr 172.21.15.71:0/3931645924 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-02-20T12:57:49.874 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:49.848+0000 7fa18219e700 1 -- 172.21.15.71:0/3931645924 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa16c005040 con 0x7fa17c0f52d0 2024-02-20T12:57:49.874 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:49.848+0000 7fa18219e700 1 --2- 172.21.15.71:0/3931645924 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa17c0f52d0 0x7fa17c0fd2b0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 rx=0x7fa16c0089f0 tx=0x7fa16c00b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:57:49.874 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:49.849+0000 7fa172ffd700 1 -- 172.21.15.71:0/3931645924 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa16c009320 con 0x7fa17c0f52d0 2024-02-20T12:57:49.874 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:49.849+0000 7fa1831a0700 1 -- 172.21.15.71:0/3931645924 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa17c0fb8d0 con 0x7fa17c0f52d0 2024-02-20T12:57:49.874 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:49.849+0000 7fa1831a0700 1 -- 172.21.15.71:0/3931645924 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa17c0fbd60 con 0x7fa17c0f52d0 2024-02-20T12:57:49.874 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:49.849+0000 7fa172ffd700 1 -- 172.21.15.71:0/3931645924 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(25 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7fa16c013ec0 con 0x7fa17c0f52d0 2024-02-20T12:57:49.874 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:49.849+0000 7fa172ffd700 1 -- 172.21.15.71:0/3931645924 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa16c014d30 con 0x7fa17c0f52d0 2024-02-20T12:57:49.875 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:49.850+0000 7fa172ffd700 1 -- 172.21.15.71:0/3931645924 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 37151+0+0 (secure 0 0 0) 0x7fa16c0095f0 con 0x7fa17c0f52d0 2024-02-20T12:57:49.875 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:49.850+0000 7fa1831a0700 1 -- 172.21.15.71:0/3931645924 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa17c0591e0 con 0x7fa17c0f52d0 2024-02-20T12:57:49.875 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:49.850+0000 7fa172ffd700 1 --2- 172.21.15.71:0/3931645924 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fa168031090 0x7fa168033550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:57:49.875 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:49.850+0000 7fa172ffd700 1 -- 172.21.15.71:0/3931645924 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1065+0+0 (secure 0 0 0) 0x7fa16c042440 con 0x7fa17c0f52d0 2024-02-20T12:57:49.875 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:49.850+0000 7fa18199d700 1 --2- 172.21.15.71:0/3931645924 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fa168031090 0x7fa168033550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:57:49.875 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:49.851+0000 7fa18199d700 1 --2- 172.21.15.71:0/3931645924 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fa168031090 0x7fa168033550 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 rx=0x7fa1780096a0 tx=0x7fa178006b40).ready entity=mgr.14156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:57:49.875 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:49.856+0000 7fa172ffd700 1 -- 172.21.15.71:0/3931645924 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+153237 (secure 0 0 0) 0x7fa16c01d7f0 con 0x7fa17c0f52d0 2024-02-20T12:57:50.066 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:50.064+0000 7fa1831a0700 1 -- 172.21.15.71:0/3931645924 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fa17c0fadc0 con 0x7fa17c0f52d0 2024-02-20T12:57:50.066 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:50.065+0000 7fa172ffd700 1 -- 172.21.15.71:0/3931645924 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7fa16c00fe40 con 0x7fa17c0f52d0 2024-02-20T12:57:50.067 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-20T12:57:50.067 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":1,"fsid":"680ee126-cfee-11ee-95bb-87774f69a715","modified":"2024-02-20T12:49:44.699059Z","created":"2024-02-20T12:49:44.699059Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi064","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:3300","nonce":0},{"type":"v1","addr":"172.21.15.64:6789","nonce":0}]},"addr":"172.21.15.64:6789/0","public_addr":"172.21.15.64:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-20T12:57:50.068 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:50.066+0000 7fa1831a0700 1 -- 172.21.15.71:0/3931645924 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fa168031090 msgr2=0x7fa168033550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:57:50.068 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:50.066+0000 7fa1831a0700 1 --2- 172.21.15.71:0/3931645924 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fa168031090 0x7fa168033550 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 rx=0x7fa1780096a0 tx=0x7fa178006b40).stop 2024-02-20T12:57:50.068 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:50.067+0000 7fa1831a0700 1 -- 172.21.15.71:0/3931645924 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa17c0f52d0 msgr2=0x7fa17c0fd2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:57:50.068 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:50.067+0000 7fa1831a0700 1 --2- 172.21.15.71:0/3931645924 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa17c0f52d0 0x7fa17c0fd2b0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 rx=0x7fa16c0089f0 tx=0x7fa16c00b040).stop 2024-02-20T12:57:50.069 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:50.067+0000 7fa1831a0700 1 -- 172.21.15.71:0/3931645924 shutdown_connections 2024-02-20T12:57:50.069 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:50.067+0000 7fa1831a0700 1 --2- 172.21.15.71:0/3931645924 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fa168031090 0x7fa168033550 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:57:50.069 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:50.067+0000 7fa1831a0700 1 --2- 172.21.15.71:0/3931645924 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa17c0f52d0 0x7fa17c0fd2b0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:57:50.069 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:50.067+0000 7fa1831a0700 1 -- 172.21.15.71:0/3931645924 >> 172.21.15.71:0/3931645924 conn(0x7fa17c0f0a30 msgr2=0x7fa17c05fee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:57:50.069 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:50.067+0000 7fa1831a0700 1 -- 172.21.15.71:0/3931645924 shutdown_connections 2024-02-20T12:57:50.069 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:50.067+0000 7fa1831a0700 1 -- 172.21.15.71:0/3931645924 wait complete. 2024-02-20T12:57:50.069 INFO:teuthology.orchestra.run.smithi071.stderr:dumped monmap epoch 1 2024-02-20T12:57:51.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:57:51 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:57:49.768647+0000 mgr.smithi064.vriyor (mgr.14156) 216 : cluster [DBG] pgmap v202: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:57:51.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:57:51 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:57:50.066853+0000 mon.smithi064 (mon.0) 230 : audit [DBG] from='client.? 172.21.15.71:0/3931645924' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-20T12:57:51.772 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-20T12:57:51.772 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/daemon-base:latest-pacific shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 680ee126-cfee-11ee-95bb-87774f69a715 -- ceph mon dump -f json 2024-02-20T12:57:53.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:57:53 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:57:51.768990+0000 mgr.smithi064.vriyor (mgr.14156) 217 : cluster [DBG] pgmap v203: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:57:53.662 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:53.660+0000 7faad843c700 1 -- 172.21.15.71:0/216601840 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faad00f6590 msgr2=0x7faad00f69b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:57:53.662 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:53.660+0000 7faad843c700 1 --2- 172.21.15.71:0/216601840 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faad00f6590 0x7faad00f69b0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 rx=0x7faac0004760 tx=0x7faac0009ad0).stop 2024-02-20T12:57:53.662 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:53.661+0000 7faad843c700 1 -- 172.21.15.71:0/216601840 shutdown_connections 2024-02-20T12:57:53.662 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:53.661+0000 7faad843c700 1 --2- 172.21.15.71:0/216601840 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faad00f6590 0x7faad00f69b0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:57:53.662 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:53.661+0000 7faad843c700 1 -- 172.21.15.71:0/216601840 >> 172.21.15.71:0/216601840 conn(0x7faad00f1cd0 msgr2=0x7faad00f4110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:57:53.662 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:53.661+0000 7faad843c700 1 -- 172.21.15.71:0/216601840 shutdown_connections 2024-02-20T12:57:53.662 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:53.661+0000 7faad843c700 1 -- 172.21.15.71:0/216601840 wait complete. 2024-02-20T12:57:53.662 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:53.661+0000 7faad843c700 1 Processor -- start 2024-02-20T12:57:53.662 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:53.661+0000 7faad843c700 1 -- start start 2024-02-20T12:57:53.662 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:53.661+0000 7faad843c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faad00f6590 0x7faad01067c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:57:53.663 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:53.661+0000 7faad843c700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faad0106cb0 con 0x7faad00f6590 2024-02-20T12:57:53.663 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:53.662+0000 7faad61d8700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faad00f6590 0x7faad01067c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:57:53.663 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:53.662+0000 7faad61d8700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faad00f6590 0x7faad01067c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.71:41684/0 (socket says 172.21.15.71:41684) 2024-02-20T12:57:53.663 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:53.662+0000 7faad61d8700 1 -- 172.21.15.71:0/2793423409 learned_addr learned my addr 172.21.15.71:0/2793423409 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-02-20T12:57:53.664 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:53.662+0000 7faad61d8700 1 -- 172.21.15.71:0/2793423409 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faac0009d70 con 0x7faad00f6590 2024-02-20T12:57:53.664 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:53.662+0000 7faad61d8700 1 --2- 172.21.15.71:0/2793423409 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faad00f6590 0x7faad01067c0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 rx=0x7faac0004170 tx=0x7faac0009ad0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:57:53.665 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:53.662+0000 7faac6ffd700 1 -- 172.21.15.71:0/2793423409 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7faac000e040 con 0x7faad00f6590 2024-02-20T12:57:53.665 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:53.662+0000 7faad843c700 1 -- 172.21.15.71:0/2793423409 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faad0106eb0 con 0x7faad00f6590 2024-02-20T12:57:53.665 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:53.662+0000 7faac6ffd700 1 -- 172.21.15.71:0/2793423409 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(25 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7faac0008ef0 con 0x7faad00f6590 2024-02-20T12:57:53.665 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:53.662+0000 7faac6ffd700 1 -- 172.21.15.71:0/2793423409 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7faac0014c90 con 0x7faad00f6590 2024-02-20T12:57:53.665 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:53.662+0000 7faad843c700 1 -- 172.21.15.71:0/2793423409 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faad0107340 con 0x7faad00f6590 2024-02-20T12:57:53.665 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:53.663+0000 7faac6ffd700 1 -- 172.21.15.71:0/2793423409 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 37151+0+0 (secure 0 0 0) 0x7faac0014420 con 0x7faad00f6590 2024-02-20T12:57:53.665 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:53.663+0000 7faad843c700 1 -- 172.21.15.71:0/2793423409 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faad0045bb0 con 0x7faad00f6590 2024-02-20T12:57:53.665 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:53.663+0000 7faac6ffd700 1 --2- 172.21.15.71:0/2793423409 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7faabc031090 0x7faabc033550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:57:53.665 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:53.663+0000 7faac6ffd700 1 -- 172.21.15.71:0/2793423409 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1065+0+0 (secure 0 0 0) 0x7faac0042240 con 0x7faad00f6590 2024-02-20T12:57:53.665 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:53.664+0000 7faad59d7700 1 --2- 172.21.15.71:0/2793423409 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7faabc031090 0x7faabc033550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:57:53.665 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:53.664+0000 7faad59d7700 1 --2- 172.21.15.71:0/2793423409 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7faabc031090 0x7faabc033550 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 rx=0x7faacc0096a0 tx=0x7faacc006b40).ready entity=mgr.14156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:57:53.669 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:53.669+0000 7faac6ffd700 1 -- 172.21.15.71:0/2793423409 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+153237 (secure 0 0 0) 0x7faac0026300 con 0x7faad00f6590 2024-02-20T12:57:53.877 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:53.876+0000 7faad843c700 1 -- 172.21.15.71:0/2793423409 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7faad01081f0 con 0x7faad00f6590 2024-02-20T12:57:53.877 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:53.877+0000 7faac6ffd700 1 -- 172.21.15.71:0/2793423409 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7faac0012b20 con 0x7faad00f6590 2024-02-20T12:57:53.877 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-20T12:57:53.878 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":1,"fsid":"680ee126-cfee-11ee-95bb-87774f69a715","modified":"2024-02-20T12:49:44.699059Z","created":"2024-02-20T12:49:44.699059Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi064","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:3300","nonce":0},{"type":"v1","addr":"172.21.15.64:6789","nonce":0}]},"addr":"172.21.15.64:6789/0","public_addr":"172.21.15.64:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-20T12:57:53.879 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:53.878+0000 7faad843c700 1 -- 172.21.15.71:0/2793423409 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7faabc031090 msgr2=0x7faabc033550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:57:53.879 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:53.878+0000 7faad843c700 1 --2- 172.21.15.71:0/2793423409 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7faabc031090 0x7faabc033550 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 rx=0x7faacc0096a0 tx=0x7faacc006b40).stop 2024-02-20T12:57:53.879 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:53.878+0000 7faad843c700 1 -- 172.21.15.71:0/2793423409 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faad00f6590 msgr2=0x7faad01067c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:57:53.879 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:53.878+0000 7faad843c700 1 --2- 172.21.15.71:0/2793423409 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faad00f6590 0x7faad01067c0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 rx=0x7faac0004170 tx=0x7faac0009ad0).stop 2024-02-20T12:57:53.879 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:53.879+0000 7faad843c700 1 -- 172.21.15.71:0/2793423409 shutdown_connections 2024-02-20T12:57:53.879 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:53.879+0000 7faad843c700 1 --2- 172.21.15.71:0/2793423409 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7faabc031090 0x7faabc033550 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:57:53.879 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:53.879+0000 7faad843c700 1 --2- 172.21.15.71:0/2793423409 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faad00f6590 0x7faad01067c0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:57:53.879 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:53.879+0000 7faad843c700 1 -- 172.21.15.71:0/2793423409 >> 172.21.15.71:0/2793423409 conn(0x7faad00f1cd0 msgr2=0x7faad00f27c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:57:53.879 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:53.879+0000 7faad843c700 1 -- 172.21.15.71:0/2793423409 shutdown_connections 2024-02-20T12:57:53.880 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:53.879+0000 7faad843c700 1 -- 172.21.15.71:0/2793423409 wait complete. 2024-02-20T12:57:53.880 INFO:teuthology.orchestra.run.smithi071.stderr:dumped monmap epoch 1 2024-02-20T12:57:54.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:57:54 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:57:53.849231+0000 mon.smithi064 (mon.0) 231 : audit [INF] from='mgr.14156 172.21.15.64:0/3284974785' entity='mgr.smithi064.vriyor' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi064.vriyor/mirror_snapshot_schedule"}]: dispatch 2024-02-20T12:57:54.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:57:54 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:57:53.849532+0000 mon.smithi064 (mon.0) 232 : audit [INF] from='mgr.14156 172.21.15.64:0/3284974785' entity='mgr.smithi064.vriyor' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi064.vriyor/trash_purge_schedule"}]: dispatch 2024-02-20T12:57:54.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:57:54 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:57:53.877815+0000 mon.smithi064 (mon.0) 233 : audit [DBG] from='client.? 172.21.15.71:0/2793423409' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-20T12:57:55.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:57:55 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:57:53.769285+0000 mgr.smithi064.vriyor (mgr.14156) 218 : cluster [DBG] pgmap v204: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:57:55.523 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-20T12:57:55.524 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/daemon-base:latest-pacific shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 680ee126-cfee-11ee-95bb-87774f69a715 -- ceph mon dump -f json 2024-02-20T12:57:57.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:57:57 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:57:55.769678+0000 mgr.smithi064.vriyor (mgr.14156) 219 : cluster [DBG] pgmap v205: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:57:57.548 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:57.536+0000 7f3bbb11c700 1 -- 172.21.15.71:0/2443123268 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3bb4060300 msgr2=0x7f3bb40f9010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:57:57.548 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:57.536+0000 7f3bbb11c700 1 --2- 172.21.15.71:0/2443123268 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3bb4060300 0x7f3bb40f9010 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 rx=0x7f3bb0009fb0 tx=0x7f3bb0009b30).stop 2024-02-20T12:57:57.548 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:57.536+0000 7f3bbb11c700 1 -- 172.21.15.71:0/2443123268 shutdown_connections 2024-02-20T12:57:57.548 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:57.536+0000 7f3bbb11c700 1 --2- 172.21.15.71:0/2443123268 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3bb4060300 0x7f3bb40f9010 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:57:57.548 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:57.536+0000 7f3bbb11c700 1 -- 172.21.15.71:0/2443123268 >> 172.21.15.71:0/2443123268 conn(0x7f3bb40f0a30 msgr2=0x7f3bb40f2e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:57:57.548 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:57.536+0000 7f3bbb11c700 1 -- 172.21.15.71:0/2443123268 shutdown_connections 2024-02-20T12:57:57.548 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:57.536+0000 7f3bbb11c700 1 -- 172.21.15.71:0/2443123268 wait complete. 2024-02-20T12:57:57.548 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:57.536+0000 7f3bbb11c700 1 Processor -- start 2024-02-20T12:57:57.548 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:57.536+0000 7f3bbb11c700 1 -- start start 2024-02-20T12:57:57.548 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:57.537+0000 7f3bbb11c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3bb4060300 0x7f3bb4106620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:57:57.549 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:57.537+0000 7f3bbb11c700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bb4106b10 con 0x7f3bb4060300 2024-02-20T12:57:57.549 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:57.537+0000 7f3bba11a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3bb4060300 0x7f3bb4106620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:57:57.549 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:57.537+0000 7f3bba11a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3bb4060300 0x7f3bb4106620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.71:42252/0 (socket says 172.21.15.71:42252) 2024-02-20T12:57:57.549 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:57.537+0000 7f3bba11a700 1 -- 172.21.15.71:0/3796029470 learned_addr learned my addr 172.21.15.71:0/3796029470 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-02-20T12:57:57.549 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:57.537+0000 7f3bba11a700 1 -- 172.21.15.71:0/3796029470 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bb0005040 con 0x7f3bb4060300 2024-02-20T12:57:57.549 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:57.537+0000 7f3bba11a700 1 --2- 172.21.15.71:0/3796029470 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3bb4060300 0x7f3bb4106620 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 rx=0x7f3bb0009f80 tx=0x7f3bb000b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:57:57.549 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:57.537+0000 7f3baaffd700 1 -- 172.21.15.71:0/3796029470 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3bb0009320 con 0x7f3bb4060300 2024-02-20T12:57:57.549 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:57.537+0000 7f3bbb11c700 1 -- 172.21.15.71:0/3796029470 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3bb4106d10 con 0x7f3bb4060300 2024-02-20T12:57:57.549 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:57.537+0000 7f3baaffd700 1 -- 172.21.15.71:0/3796029470 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(25 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f3bb0013680 con 0x7f3bb4060300 2024-02-20T12:57:57.549 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:57.537+0000 7f3baaffd700 1 -- 172.21.15.71:0/3796029470 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3bb001eda0 con 0x7f3bb4060300 2024-02-20T12:57:57.549 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:57.537+0000 7f3bbb11c700 1 -- 172.21.15.71:0/3796029470 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3bb41071a0 con 0x7f3bb4060300 2024-02-20T12:57:57.549 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:57.538+0000 7f3baaffd700 1 -- 172.21.15.71:0/3796029470 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 37151+0+0 (secure 0 0 0) 0x7f3bb001e420 con 0x7f3bb4060300 2024-02-20T12:57:57.549 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:57.538+0000 7f3baaffd700 1 --2- 172.21.15.71:0/3796029470 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f3ba00310e0 0x7f3ba00335a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:57:57.549 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:57.538+0000 7f3bbb11c700 1 -- 172.21.15.71:0/3796029470 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3bb4045bb0 con 0x7f3bb4060300 2024-02-20T12:57:57.549 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:57.538+0000 7f3baaffd700 1 -- 172.21.15.71:0/3796029470 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1065+0+0 (secure 0 0 0) 0x7f3bb0042470 con 0x7f3bb4060300 2024-02-20T12:57:57.550 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:57.539+0000 7f3bb9919700 1 --2- 172.21.15.71:0/3796029470 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f3ba00310e0 0x7f3ba00335a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:57:57.550 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:57.539+0000 7f3bb9919700 1 --2- 172.21.15.71:0/3796029470 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f3ba00310e0 0x7f3ba00335a0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 rx=0x7f3ba40096a0 tx=0x7f3ba4006b40).ready entity=mgr.14156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:57:57.550 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:57.543+0000 7f3baaffd700 1 -- 172.21.15.71:0/3796029470 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+153237 (secure 0 0 0) 0x7f3bb001d380 con 0x7f3bb4060300 2024-02-20T12:57:57.753 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:57.753+0000 7f3bbb11c700 1 -- 172.21.15.71:0/3796029470 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f3bb4108020 con 0x7f3bb4060300 2024-02-20T12:57:57.754 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:57.753+0000 7f3baaffd700 1 -- 172.21.15.71:0/3796029470 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7f3bb0004490 con 0x7f3bb4060300 2024-02-20T12:57:57.754 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-20T12:57:57.754 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":1,"fsid":"680ee126-cfee-11ee-95bb-87774f69a715","modified":"2024-02-20T12:49:44.699059Z","created":"2024-02-20T12:49:44.699059Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi064","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:3300","nonce":0},{"type":"v1","addr":"172.21.15.64:6789","nonce":0}]},"addr":"172.21.15.64:6789/0","public_addr":"172.21.15.64:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-20T12:57:57.755 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:57.755+0000 7f3bbb11c700 1 -- 172.21.15.71:0/3796029470 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f3ba00310e0 msgr2=0x7f3ba00335a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:57:57.756 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:57.755+0000 7f3bbb11c700 1 --2- 172.21.15.71:0/3796029470 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f3ba00310e0 0x7f3ba00335a0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 rx=0x7f3ba40096a0 tx=0x7f3ba4006b40).stop 2024-02-20T12:57:57.756 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:57.755+0000 7f3bbb11c700 1 -- 172.21.15.71:0/3796029470 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3bb4060300 msgr2=0x7f3bb4106620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:57:57.756 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:57.755+0000 7f3bbb11c700 1 --2- 172.21.15.71:0/3796029470 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3bb4060300 0x7f3bb4106620 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 rx=0x7f3bb0009f80 tx=0x7f3bb000b040).stop 2024-02-20T12:57:57.756 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:57.755+0000 7f3bbb11c700 1 -- 172.21.15.71:0/3796029470 shutdown_connections 2024-02-20T12:57:57.756 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:57.755+0000 7f3bbb11c700 1 --2- 172.21.15.71:0/3796029470 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f3ba00310e0 0x7f3ba00335a0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:57:57.756 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:57.755+0000 7f3bbb11c700 1 --2- 172.21.15.71:0/3796029470 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3bb4060300 0x7f3bb4106620 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:57:57.756 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:57.755+0000 7f3bbb11c700 1 -- 172.21.15.71:0/3796029470 >> 172.21.15.71:0/3796029470 conn(0x7f3bb40f0a30 msgr2=0x7f3bb40f1540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:57:57.756 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:57.755+0000 7f3bbb11c700 1 -- 172.21.15.71:0/3796029470 shutdown_connections 2024-02-20T12:57:57.756 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:57:57.755+0000 7f3bbb11c700 1 -- 172.21.15.71:0/3796029470 wait complete. 2024-02-20T12:57:57.757 INFO:teuthology.orchestra.run.smithi071.stderr:dumped monmap epoch 1 2024-02-20T12:57:58.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:57:58 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:57:57.754172+0000 mon.smithi064 (mon.0) 234 : audit [DBG] from='client.? 172.21.15.71:0/3796029470' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-20T12:57:59.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:57:59 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:57:57.770187+0000 mgr.smithi064.vriyor (mgr.14156) 220 : cluster [DBG] pgmap v206: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:57:59.543 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-20T12:57:59.544 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/daemon-base:latest-pacific shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 680ee126-cfee-11ee-95bb-87774f69a715 -- ceph mon dump -f json 2024-02-20T12:58:01.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:58:01 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:57:59.770638+0000 mgr.smithi064.vriyor (mgr.14156) 221 : cluster [DBG] pgmap v207: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:58:01.499 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:01.497+0000 7fc6f8e77700 1 -- 172.21.15.71:0/1247436712 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc6f40602c0 msgr2=0x7fc6f40f9200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:58:01.499 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:01.497+0000 7fc6f8e77700 1 --2- 172.21.15.71:0/1247436712 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc6f40602c0 0x7fc6f40f9200 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 rx=0x7fc6dc009fb0 tx=0x7fc6dc009b30).stop 2024-02-20T12:58:01.499 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:01.497+0000 7fc6f8e77700 1 -- 172.21.15.71:0/1247436712 shutdown_connections 2024-02-20T12:58:01.499 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:01.497+0000 7fc6f8e77700 1 --2- 172.21.15.71:0/1247436712 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc6f40602c0 0x7fc6f40f9200 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:58:01.499 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:01.497+0000 7fc6f8e77700 1 -- 172.21.15.71:0/1247436712 >> 172.21.15.71:0/1247436712 conn(0x7fc6f40f0be0 msgr2=0x7fc6f40f3000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:58:01.499 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:01.497+0000 7fc6f8e77700 1 -- 172.21.15.71:0/1247436712 shutdown_connections 2024-02-20T12:58:01.499 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:01.497+0000 7fc6f8e77700 1 -- 172.21.15.71:0/1247436712 wait complete. 2024-02-20T12:58:01.500 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:01.498+0000 7fc6f8e77700 1 Processor -- start 2024-02-20T12:58:01.500 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:01.498+0000 7fc6f8e77700 1 -- start start 2024-02-20T12:58:01.500 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:01.498+0000 7fc6f8e77700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc6f40602c0 0x7fc6f4106730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:58:01.500 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:01.498+0000 7fc6f8e77700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6f4106c20 con 0x7fc6f40602c0 2024-02-20T12:58:01.500 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:01.498+0000 7fc6f259c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc6f40602c0 0x7fc6f4106730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:58:01.500 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:01.498+0000 7fc6f259c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc6f40602c0 0x7fc6f4106730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.71:42268/0 (socket says 172.21.15.71:42268) 2024-02-20T12:58:01.500 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:01.498+0000 7fc6f259c700 1 -- 172.21.15.71:0/236649187 learned_addr learned my addr 172.21.15.71:0/236649187 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-02-20T12:58:01.500 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:01.498+0000 7fc6f259c700 1 -- 172.21.15.71:0/236649187 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6dc005040 con 0x7fc6f40602c0 2024-02-20T12:58:01.500 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:01.499+0000 7fc6f259c700 1 --2- 172.21.15.71:0/236649187 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc6f40602c0 0x7fc6f4106730 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 rx=0x7fc6dc013fd0 tx=0x7fc6dc009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:58:01.500 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:01.499+0000 7fc6eaffd700 1 -- 172.21.15.71:0/236649187 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc6dc018070 con 0x7fc6f40602c0 2024-02-20T12:58:01.501 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:01.499+0000 7fc6f8e77700 1 -- 172.21.15.71:0/236649187 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6f4106e20 con 0x7fc6f40602c0 2024-02-20T12:58:01.501 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:01.499+0000 7fc6eaffd700 1 -- 172.21.15.71:0/236649187 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(25 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7fc6dc007b10 con 0x7fc6f40602c0 2024-02-20T12:58:01.501 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:01.499+0000 7fc6eaffd700 1 -- 172.21.15.71:0/236649187 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc6dc01c3c0 con 0x7fc6f40602c0 2024-02-20T12:58:01.501 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:01.499+0000 7fc6f8e77700 1 -- 172.21.15.71:0/236649187 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6f41072b0 con 0x7fc6f40602c0 2024-02-20T12:58:01.502 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:01.500+0000 7fc6eaffd700 1 -- 172.21.15.71:0/236649187 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 37151+0+0 (secure 0 0 0) 0x7fc6dc007c80 con 0x7fc6f40602c0 2024-02-20T12:58:01.502 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:01.500+0000 7fc6f8e77700 1 -- 172.21.15.71:0/236649187 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6f4045bb0 con 0x7fc6f40602c0 2024-02-20T12:58:01.502 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:01.500+0000 7fc6eaffd700 1 --2- 172.21.15.71:0/236649187 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fc6e0031090 0x7fc6e0033550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:58:01.502 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:01.500+0000 7fc6eaffd700 1 -- 172.21.15.71:0/236649187 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1065+0+0 (secure 0 0 0) 0x7fc6dc042990 con 0x7fc6f40602c0 2024-02-20T12:58:01.502 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:01.500+0000 7fc6f1d9b700 1 --2- 172.21.15.71:0/236649187 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fc6e0031090 0x7fc6e0033550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:58:01.502 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:01.500+0000 7fc6f1d9b700 1 --2- 172.21.15.71:0/236649187 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fc6e0031090 0x7fc6e0033550 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 rx=0x7fc6e40096a0 tx=0x7fc6e4006b40).ready entity=mgr.14156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:58:01.505 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:01.505+0000 7fc6eaffd700 1 -- 172.21.15.71:0/236649187 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+153237 (secure 0 0 0) 0x7fc6dc01cb60 con 0x7fc6f40602c0 2024-02-20T12:58:01.710 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:01.709+0000 7fc6f8e77700 1 -- 172.21.15.71:0/236649187 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fc6f4108160 con 0x7fc6f40602c0 2024-02-20T12:58:01.710 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:01.710+0000 7fc6eaffd700 1 -- 172.21.15.71:0/236649187 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7fc6dc025b80 con 0x7fc6f40602c0 2024-02-20T12:58:01.710 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-20T12:58:01.710 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":1,"fsid":"680ee126-cfee-11ee-95bb-87774f69a715","modified":"2024-02-20T12:49:44.699059Z","created":"2024-02-20T12:49:44.699059Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi064","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:3300","nonce":0},{"type":"v1","addr":"172.21.15.64:6789","nonce":0}]},"addr":"172.21.15.64:6789/0","public_addr":"172.21.15.64:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-20T12:58:01.712 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:01.711+0000 7fc6f8e77700 1 -- 172.21.15.71:0/236649187 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fc6e0031090 msgr2=0x7fc6e0033550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:58:01.712 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:01.711+0000 7fc6f8e77700 1 --2- 172.21.15.71:0/236649187 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fc6e0031090 0x7fc6e0033550 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 rx=0x7fc6e40096a0 tx=0x7fc6e4006b40).stop 2024-02-20T12:58:01.712 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:01.711+0000 7fc6f8e77700 1 -- 172.21.15.71:0/236649187 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc6f40602c0 msgr2=0x7fc6f4106730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:58:01.712 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:01.711+0000 7fc6f8e77700 1 --2- 172.21.15.71:0/236649187 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc6f40602c0 0x7fc6f4106730 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 rx=0x7fc6dc013fd0 tx=0x7fc6dc009b30).stop 2024-02-20T12:58:01.712 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:01.711+0000 7fc6f8e77700 1 -- 172.21.15.71:0/236649187 shutdown_connections 2024-02-20T12:58:01.712 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:01.711+0000 7fc6f8e77700 1 --2- 172.21.15.71:0/236649187 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fc6e0031090 0x7fc6e0033550 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:58:01.712 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:01.711+0000 7fc6f8e77700 1 --2- 172.21.15.71:0/236649187 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc6f40602c0 0x7fc6f4106730 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:58:01.713 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:01.712+0000 7fc6f8e77700 1 -- 172.21.15.71:0/236649187 >> 172.21.15.71:0/236649187 conn(0x7fc6f40f0be0 msgr2=0x7fc6f40f1630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:58:01.713 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:01.712+0000 7fc6f8e77700 1 -- 172.21.15.71:0/236649187 shutdown_connections 2024-02-20T12:58:01.713 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:01.712+0000 7fc6f8e77700 1 -- 172.21.15.71:0/236649187 wait complete. 2024-02-20T12:58:01.713 INFO:teuthology.orchestra.run.smithi071.stderr:dumped monmap epoch 1 2024-02-20T12:58:02.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:58:02 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:58:01.710829+0000 mon.smithi064 (mon.0) 235 : audit [DBG] from='client.? 172.21.15.71:0/236649187' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-20T12:58:03.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:58:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:58:01.770917+0000 mgr.smithi064.vriyor (mgr.14156) 222 : cluster [DBG] pgmap v208: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:58:03.506 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-20T12:58:03.506 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/daemon-base:latest-pacific shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 680ee126-cfee-11ee-95bb-87774f69a715 -- ceph mon dump -f json 2024-02-20T12:58:05.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:58:05 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:58:03.771237+0000 mgr.smithi064.vriyor (mgr.14156) 223 : cluster [DBG] pgmap v209: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:58:05.398 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:05.396+0000 7f1593fff700 1 -- 172.21.15.71:0/1743981962 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1594060300 msgr2=0x7f15940f9010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:58:05.398 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:05.396+0000 7f1593fff700 1 --2- 172.21.15.71:0/1743981962 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1594060300 0x7f15940f9010 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 rx=0x7f1584009fb0 tx=0x7f1584009b30).stop 2024-02-20T12:58:05.398 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:05.396+0000 7f1593fff700 1 -- 172.21.15.71:0/1743981962 shutdown_connections 2024-02-20T12:58:05.398 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:05.396+0000 7f1593fff700 1 --2- 172.21.15.71:0/1743981962 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1594060300 0x7f15940f9010 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:58:05.398 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:05.396+0000 7f1593fff700 1 -- 172.21.15.71:0/1743981962 >> 172.21.15.71:0/1743981962 conn(0x7f15940f0a30 msgr2=0x7f15940f2e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:58:05.398 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:05.396+0000 7f1593fff700 1 -- 172.21.15.71:0/1743981962 shutdown_connections 2024-02-20T12:58:05.398 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:05.396+0000 7f1593fff700 1 -- 172.21.15.71:0/1743981962 wait complete. 2024-02-20T12:58:05.398 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:05.396+0000 7f1593fff700 1 Processor -- start 2024-02-20T12:58:05.399 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:05.396+0000 7f1593fff700 1 -- start start 2024-02-20T12:58:05.399 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:05.397+0000 7f1593fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1594060300 0x7f15940f6a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:58:05.399 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:05.397+0000 7f1593fff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15940f4f10 con 0x7f1594060300 2024-02-20T12:58:05.399 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:05.397+0000 7f1592ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1594060300 0x7f15940f6a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:58:05.399 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:05.397+0000 7f1592ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1594060300 0x7f15940f6a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.71:42284/0 (socket says 172.21.15.71:42284) 2024-02-20T12:58:05.399 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:05.397+0000 7f1592ffd700 1 -- 172.21.15.71:0/1692486737 learned_addr learned my addr 172.21.15.71:0/1692486737 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-02-20T12:58:05.399 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:05.397+0000 7f1592ffd700 1 -- 172.21.15.71:0/1692486737 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1584005040 con 0x7f1594060300 2024-02-20T12:58:05.399 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:05.397+0000 7f1592ffd700 1 --2- 172.21.15.71:0/1692486737 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1594060300 0x7f15940f6a90 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 rx=0x7f1584009f80 tx=0x7f158400b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:58:05.399 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:05.397+0000 7f157b7fe700 1 -- 172.21.15.71:0/1692486737 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1584009320 con 0x7f1594060300 2024-02-20T12:58:05.399 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:05.397+0000 7f1593fff700 1 -- 172.21.15.71:0/1692486737 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15940f50b0 con 0x7f1594060300 2024-02-20T12:58:05.399 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:05.397+0000 7f157b7fe700 1 -- 172.21.15.71:0/1692486737 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(25 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f1584013680 con 0x7f1594060300 2024-02-20T12:58:05.399 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:05.397+0000 7f157b7fe700 1 -- 172.21.15.71:0/1692486737 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f158401eda0 con 0x7f1594060300 2024-02-20T12:58:05.399 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:05.397+0000 7f1593fff700 1 -- 172.21.15.71:0/1692486737 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15940f5540 con 0x7f1594060300 2024-02-20T12:58:05.400 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:05.398+0000 7f157b7fe700 1 -- 172.21.15.71:0/1692486737 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 37151+0+0 (secure 0 0 0) 0x7f158401e420 con 0x7f1594060300 2024-02-20T12:58:05.401 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:05.398+0000 7f157b7fe700 1 --2- 172.21.15.71:0/1692486737 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f157c031090 0x7f157c033550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:58:05.401 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:05.398+0000 7f157b7fe700 1 -- 172.21.15.71:0/1692486737 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1065+0+0 (secure 0 0 0) 0x7f1584042360 con 0x7f1594060300 2024-02-20T12:58:05.401 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:05.398+0000 7f1593fff700 1 -- 172.21.15.71:0/1692486737 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15940591e0 con 0x7f1594060300 2024-02-20T12:58:05.401 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:05.399+0000 7f15927fc700 1 --2- 172.21.15.71:0/1692486737 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f157c031090 0x7f157c033550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:58:05.401 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:05.399+0000 7f15927fc700 1 --2- 172.21.15.71:0/1692486737 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f157c031090 0x7f157c033550 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 rx=0x7f15880096a0 tx=0x7f1588006b40).ready entity=mgr.14156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:58:05.404 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:05.403+0000 7f157b7fe700 1 -- 172.21.15.71:0/1692486737 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+153237 (secure 0 0 0) 0x7f158401e700 con 0x7f1594060300 2024-02-20T12:58:05.610 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:05.610+0000 7f1593fff700 1 -- 172.21.15.71:0/1692486737 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f1594045bb0 con 0x7f1594060300 2024-02-20T12:58:05.611 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:05.611+0000 7f157b7fe700 1 -- 172.21.15.71:0/1692486737 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7f1584004490 con 0x7f1594060300 2024-02-20T12:58:05.611 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-20T12:58:05.611 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":1,"fsid":"680ee126-cfee-11ee-95bb-87774f69a715","modified":"2024-02-20T12:49:44.699059Z","created":"2024-02-20T12:49:44.699059Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi064","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:3300","nonce":0},{"type":"v1","addr":"172.21.15.64:6789","nonce":0}]},"addr":"172.21.15.64:6789/0","public_addr":"172.21.15.64:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-20T12:58:05.613 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:05.612+0000 7f1593fff700 1 -- 172.21.15.71:0/1692486737 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f157c031090 msgr2=0x7f157c033550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:58:05.613 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:05.612+0000 7f1593fff700 1 --2- 172.21.15.71:0/1692486737 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f157c031090 0x7f157c033550 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 rx=0x7f15880096a0 tx=0x7f1588006b40).stop 2024-02-20T12:58:05.613 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:05.612+0000 7f1593fff700 1 -- 172.21.15.71:0/1692486737 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1594060300 msgr2=0x7f15940f6a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:58:05.613 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:05.612+0000 7f1593fff700 1 --2- 172.21.15.71:0/1692486737 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1594060300 0x7f15940f6a90 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 rx=0x7f1584009f80 tx=0x7f158400b040).stop 2024-02-20T12:58:05.613 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:05.612+0000 7f1593fff700 1 -- 172.21.15.71:0/1692486737 shutdown_connections 2024-02-20T12:58:05.613 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:05.612+0000 7f1593fff700 1 --2- 172.21.15.71:0/1692486737 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f157c031090 0x7f157c033550 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:58:05.613 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:05.612+0000 7f1593fff700 1 --2- 172.21.15.71:0/1692486737 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1594060300 0x7f15940f6a90 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:58:05.614 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:05.612+0000 7f1593fff700 1 -- 172.21.15.71:0/1692486737 >> 172.21.15.71:0/1692486737 conn(0x7f15940f0a30 msgr2=0x7f15940f1540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:58:05.614 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:05.612+0000 7f1593fff700 1 -- 172.21.15.71:0/1692486737 shutdown_connections 2024-02-20T12:58:05.614 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:05.612+0000 7f1593fff700 1 -- 172.21.15.71:0/1692486737 wait complete. 2024-02-20T12:58:05.614 INFO:teuthology.orchestra.run.smithi071.stderr:dumped monmap epoch 1 2024-02-20T12:58:06.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:58:06 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:58:05.611619+0000 mon.smithi064 (mon.0) 236 : audit [DBG] from='client.? 172.21.15.71:0/1692486737' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-20T12:58:07.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:58:07 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:58:05.771672+0000 mgr.smithi064.vriyor (mgr.14156) 224 : cluster [DBG] pgmap v210: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:58:07.374 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-20T12:58:07.375 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/daemon-base:latest-pacific shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 680ee126-cfee-11ee-95bb-87774f69a715 -- ceph mon dump -f json 2024-02-20T12:58:09.201 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:09.199+0000 7fad11959700 1 -- 172.21.15.71:0/2022873462 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad0c0f6590 msgr2=0x7fad0c0f69b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:58:09.201 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:09.199+0000 7fad11959700 1 --2- 172.21.15.71:0/2022873462 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad0c0f6590 0x7fad0c0f69b0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 rx=0x7facfc009fb0 tx=0x7facfc009b30).stop 2024-02-20T12:58:09.201 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:09.199+0000 7fad11959700 1 -- 172.21.15.71:0/2022873462 shutdown_connections 2024-02-20T12:58:09.201 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:09.199+0000 7fad11959700 1 --2- 172.21.15.71:0/2022873462 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad0c0f6590 0x7fad0c0f69b0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:58:09.201 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:09.199+0000 7fad11959700 1 -- 172.21.15.71:0/2022873462 >> 172.21.15.71:0/2022873462 conn(0x7fad0c0f1cf0 msgr2=0x7fad0c0f4110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:58:09.202 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:09.199+0000 7fad11959700 1 -- 172.21.15.71:0/2022873462 shutdown_connections 2024-02-20T12:58:09.202 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:09.199+0000 7fad11959700 1 -- 172.21.15.71:0/2022873462 wait complete. 2024-02-20T12:58:09.202 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:09.200+0000 7fad11959700 1 Processor -- start 2024-02-20T12:58:09.202 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:09.200+0000 7fad11959700 1 -- start start 2024-02-20T12:58:09.202 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:09.200+0000 7fad11959700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad0c0f6590 0x7fad0c06ba80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:58:09.202 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:09.200+0000 7fad11959700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad0c069f00 con 0x7fad0c0f6590 2024-02-20T12:58:09.202 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:09.201+0000 7fad0affd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad0c0f6590 0x7fad0c06ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:58:09.202 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:09.201+0000 7fad0affd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad0c0f6590 0x7fad0c06ba80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.71:35228/0 (socket says 172.21.15.71:35228) 2024-02-20T12:58:09.202 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:09.201+0000 7fad0affd700 1 -- 172.21.15.71:0/1460745163 learned_addr learned my addr 172.21.15.71:0/1460745163 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-02-20T12:58:09.202 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:09.201+0000 7fad0affd700 1 -- 172.21.15.71:0/1460745163 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facfc005040 con 0x7fad0c0f6590 2024-02-20T12:58:09.202 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:09.201+0000 7fad0affd700 1 --2- 172.21.15.71:0/1460745163 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad0c0f6590 0x7fad0c06ba80 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 rx=0x7facfc0089f0 tx=0x7facfc00b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:58:09.203 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:09.201+0000 7facf37fe700 1 -- 172.21.15.71:0/1460745163 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7facfc009320 con 0x7fad0c0f6590 2024-02-20T12:58:09.204 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:09.201+0000 7fad11959700 1 -- 172.21.15.71:0/1460745163 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad0c06a0a0 con 0x7fad0c0f6590 2024-02-20T12:58:09.204 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:09.201+0000 7facf37fe700 1 -- 172.21.15.71:0/1460745163 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(25 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7facfc013ec0 con 0x7fad0c0f6590 2024-02-20T12:58:09.204 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:09.201+0000 7facf37fe700 1 -- 172.21.15.71:0/1460745163 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7facfc014d30 con 0x7fad0c0f6590 2024-02-20T12:58:09.204 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:09.201+0000 7fad11959700 1 -- 172.21.15.71:0/1460745163 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad0c06a530 con 0x7fad0c0f6590 2024-02-20T12:58:09.204 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:09.202+0000 7facf37fe700 1 -- 172.21.15.71:0/1460745163 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 37151+0+0 (secure 0 0 0) 0x7facfc0095f0 con 0x7fad0c0f6590 2024-02-20T12:58:09.204 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:09.202+0000 7facf37fe700 1 --2- 172.21.15.71:0/1460745163 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7facf4031090 0x7facf4033550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:58:09.204 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:09.202+0000 7facf37fe700 1 -- 172.21.15.71:0/1460745163 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1065+0+0 (secure 0 0 0) 0x7facfc0423e0 con 0x7fad0c0f6590 2024-02-20T12:58:09.204 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:09.202+0000 7fad11959700 1 -- 172.21.15.71:0/1460745163 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad0c0591e0 con 0x7fad0c0f6590 2024-02-20T12:58:09.204 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:09.203+0000 7fad0a7fc700 1 --2- 172.21.15.71:0/1460745163 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7facf4031090 0x7facf4033550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:58:09.204 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:09.203+0000 7fad0a7fc700 1 --2- 172.21.15.71:0/1460745163 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7facf4031090 0x7facf4033550 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 rx=0x7fad000096a0 tx=0x7fad00006b40).ready entity=mgr.14156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:58:09.208 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:09.207+0000 7facf37fe700 1 -- 172.21.15.71:0/1460745163 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+153237 (secure 0 0 0) 0x7facfc01dd70 con 0x7fad0c0f6590 2024-02-20T12:58:09.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:58:09 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:58:07.772008+0000 mgr.smithi064.vriyor (mgr.14156) 225 : cluster [DBG] pgmap v211: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:58:09.414 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:09.414+0000 7fad11959700 1 -- 172.21.15.71:0/1460745163 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fad0c06ab90 con 0x7fad0c0f6590 2024-02-20T12:58:09.415 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:09.414+0000 7facf37fe700 1 -- 172.21.15.71:0/1460745163 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7facfc02b020 con 0x7fad0c0f6590 2024-02-20T12:58:09.415 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-20T12:58:09.415 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":1,"fsid":"680ee126-cfee-11ee-95bb-87774f69a715","modified":"2024-02-20T12:49:44.699059Z","created":"2024-02-20T12:49:44.699059Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi064","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:3300","nonce":0},{"type":"v1","addr":"172.21.15.64:6789","nonce":0}]},"addr":"172.21.15.64:6789/0","public_addr":"172.21.15.64:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-20T12:58:09.416 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:09.416+0000 7fad11959700 1 -- 172.21.15.71:0/1460745163 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7facf4031090 msgr2=0x7facf4033550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:58:09.416 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:09.416+0000 7fad11959700 1 --2- 172.21.15.71:0/1460745163 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7facf4031090 0x7facf4033550 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 rx=0x7fad000096a0 tx=0x7fad00006b40).stop 2024-02-20T12:58:09.417 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:09.416+0000 7fad11959700 1 -- 172.21.15.71:0/1460745163 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad0c0f6590 msgr2=0x7fad0c06ba80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:58:09.417 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:09.416+0000 7fad11959700 1 --2- 172.21.15.71:0/1460745163 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad0c0f6590 0x7fad0c06ba80 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 rx=0x7facfc0089f0 tx=0x7facfc00b040).stop 2024-02-20T12:58:09.417 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:09.416+0000 7fad11959700 1 -- 172.21.15.71:0/1460745163 shutdown_connections 2024-02-20T12:58:09.417 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:09.416+0000 7fad11959700 1 --2- 172.21.15.71:0/1460745163 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7facf4031090 0x7facf4033550 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:58:09.417 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:09.416+0000 7fad11959700 1 --2- 172.21.15.71:0/1460745163 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad0c0f6590 0x7fad0c06ba80 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:58:09.417 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:09.416+0000 7fad11959700 1 -- 172.21.15.71:0/1460745163 >> 172.21.15.71:0/1460745163 conn(0x7fad0c0f1cf0 msgr2=0x7fad0c0f27e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:58:09.417 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:09.416+0000 7fad11959700 1 -- 172.21.15.71:0/1460745163 shutdown_connections 2024-02-20T12:58:09.417 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:09.416+0000 7fad11959700 1 -- 172.21.15.71:0/1460745163 wait complete. 2024-02-20T12:58:09.417 INFO:teuthology.orchestra.run.smithi071.stderr:dumped monmap epoch 1 2024-02-20T12:58:10.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:58:10 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:58:09.415271+0000 mon.smithi064 (mon.0) 237 : audit [DBG] from='client.? 172.21.15.71:0/1460745163' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-20T12:58:11.173 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-20T12:58:11.174 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/daemon-base:latest-pacific shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 680ee126-cfee-11ee-95bb-87774f69a715 -- ceph mon dump -f json 2024-02-20T12:58:11.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:58:11 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:58:09.772435+0000 mgr.smithi064.vriyor (mgr.14156) 226 : cluster [DBG] pgmap v212: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:58:13.129 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:13.127+0000 7f259f373700 1 -- 172.21.15.71:0/674858713 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f25980f52d0 msgr2=0x7f25980f56f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:58:13.129 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:13.127+0000 7f259f373700 1 --2- 172.21.15.71:0/674858713 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f25980f52d0 0x7f25980f56f0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 rx=0x7f2588009fb0 tx=0x7f2588009b30).stop 2024-02-20T12:58:13.130 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:13.127+0000 7f259f373700 1 -- 172.21.15.71:0/674858713 shutdown_connections 2024-02-20T12:58:13.130 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:13.127+0000 7f259f373700 1 --2- 172.21.15.71:0/674858713 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f25980f52d0 0x7f25980f56f0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:58:13.130 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:13.127+0000 7f259f373700 1 -- 172.21.15.71:0/674858713 >> 172.21.15.71:0/674858713 conn(0x7f25980f0a30 msgr2=0x7f25980f2e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:58:13.130 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:13.127+0000 7f259f373700 1 -- 172.21.15.71:0/674858713 shutdown_connections 2024-02-20T12:58:13.130 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:13.127+0000 7f259f373700 1 -- 172.21.15.71:0/674858713 wait complete. 2024-02-20T12:58:13.130 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:13.128+0000 7f259f373700 1 Processor -- start 2024-02-20T12:58:13.130 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:13.128+0000 7f259f373700 1 -- start start 2024-02-20T12:58:13.130 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:13.128+0000 7f259f373700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f25980f52d0 0x7f2598109330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:58:13.130 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:13.128+0000 7f259f373700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2598060300 con 0x7f25980f52d0 2024-02-20T12:58:13.130 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:13.128+0000 7f259e371700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f25980f52d0 0x7f2598109330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:58:13.130 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:13.128+0000 7f259e371700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f25980f52d0 0x7f2598109330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.71:35248/0 (socket says 172.21.15.71:35248) 2024-02-20T12:58:13.130 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:13.128+0000 7f259e371700 1 -- 172.21.15.71:0/1463812651 learned_addr learned my addr 172.21.15.71:0/1463812651 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-02-20T12:58:13.130 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:13.128+0000 7f259e371700 1 -- 172.21.15.71:0/1463812651 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2588005040 con 0x7f25980f52d0 2024-02-20T12:58:13.131 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:13.129+0000 7f259e371700 1 --2- 172.21.15.71:0/1463812651 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f25980f52d0 0x7f2598109330 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 rx=0x7f25880089f0 tx=0x7f258800b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:58:13.131 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:13.129+0000 7f258effd700 1 -- 172.21.15.71:0/1463812651 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2588009320 con 0x7f25980f52d0 2024-02-20T12:58:13.131 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:13.129+0000 7f259f373700 1 -- 172.21.15.71:0/1463812651 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2598060500 con 0x7f25980f52d0 2024-02-20T12:58:13.131 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:13.129+0000 7f258effd700 1 -- 172.21.15.71:0/1463812651 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(25 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f2588013ec0 con 0x7f25980f52d0 2024-02-20T12:58:13.131 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:13.129+0000 7f258effd700 1 -- 172.21.15.71:0/1463812651 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2588014d30 con 0x7f25980f52d0 2024-02-20T12:58:13.131 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:13.129+0000 7f259f373700 1 -- 172.21.15.71:0/1463812651 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25981099e0 con 0x7f25980f52d0 2024-02-20T12:58:13.132 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:13.130+0000 7f258effd700 1 -- 172.21.15.71:0/1463812651 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 37151+0+0 (secure 0 0 0) 0x7f2588014420 con 0x7f25980f52d0 2024-02-20T12:58:13.132 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:13.130+0000 7f259f373700 1 -- 172.21.15.71:0/1463812651 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25980591e0 con 0x7f25980f52d0 2024-02-20T12:58:13.132 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:13.130+0000 7f258effd700 1 --2- 172.21.15.71:0/1463812651 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f2584031090 0x7f2584033550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:58:13.132 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:13.130+0000 7f258effd700 1 -- 172.21.15.71:0/1463812651 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1065+0+0 (secure 0 0 0) 0x7f25880424a0 con 0x7f25980f52d0 2024-02-20T12:58:13.132 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:13.130+0000 7f259db70700 1 --2- 172.21.15.71:0/1463812651 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f2584031090 0x7f2584033550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:58:13.132 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:13.131+0000 7f259db70700 1 --2- 172.21.15.71:0/1463812651 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f2584031090 0x7f2584033550 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 rx=0x7f25940096a0 tx=0x7f2594006b40).ready entity=mgr.14156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:58:13.135 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:13.135+0000 7f258effd700 1 -- 172.21.15.71:0/1463812651 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+153237 (secure 0 0 0) 0x7f258801d7f0 con 0x7f25980f52d0 2024-02-20T12:58:13.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:58:13 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:58:11.772801+0000 mgr.smithi064.vriyor (mgr.14156) 227 : cluster [DBG] pgmap v213: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:58:13.342 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:13.342+0000 7f259f373700 1 -- 172.21.15.71:0/1463812651 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f25980fadc0 con 0x7f25980f52d0 2024-02-20T12:58:13.343 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:13.342+0000 7f258effd700 1 -- 172.21.15.71:0/1463812651 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7f258801d7f0 con 0x7f25980f52d0 2024-02-20T12:58:13.343 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-20T12:58:13.343 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":1,"fsid":"680ee126-cfee-11ee-95bb-87774f69a715","modified":"2024-02-20T12:49:44.699059Z","created":"2024-02-20T12:49:44.699059Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi064","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:3300","nonce":0},{"type":"v1","addr":"172.21.15.64:6789","nonce":0}]},"addr":"172.21.15.64:6789/0","public_addr":"172.21.15.64:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-20T12:58:13.345 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:13.344+0000 7f259f373700 1 -- 172.21.15.71:0/1463812651 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f2584031090 msgr2=0x7f2584033550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:58:13.345 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:13.344+0000 7f259f373700 1 --2- 172.21.15.71:0/1463812651 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f2584031090 0x7f2584033550 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 rx=0x7f25940096a0 tx=0x7f2594006b40).stop 2024-02-20T12:58:13.345 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:13.344+0000 7f259f373700 1 -- 172.21.15.71:0/1463812651 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f25980f52d0 msgr2=0x7f2598109330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:58:13.345 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:13.344+0000 7f259f373700 1 --2- 172.21.15.71:0/1463812651 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f25980f52d0 0x7f2598109330 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 rx=0x7f25880089f0 tx=0x7f258800b040).stop 2024-02-20T12:58:13.345 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:13.344+0000 7f259f373700 1 -- 172.21.15.71:0/1463812651 shutdown_connections 2024-02-20T12:58:13.345 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:13.344+0000 7f259f373700 1 --2- 172.21.15.71:0/1463812651 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f2584031090 0x7f2584033550 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:58:13.346 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:13.344+0000 7f259f373700 1 --2- 172.21.15.71:0/1463812651 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f25980f52d0 0x7f2598109330 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:58:13.346 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:13.344+0000 7f259f373700 1 -- 172.21.15.71:0/1463812651 >> 172.21.15.71:0/1463812651 conn(0x7f25980f0a30 msgr2=0x7f25980f1550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:58:13.346 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:13.344+0000 7f259f373700 1 -- 172.21.15.71:0/1463812651 shutdown_connections 2024-02-20T12:58:13.346 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:13.344+0000 7f259f373700 1 -- 172.21.15.71:0/1463812651 wait complete. 2024-02-20T12:58:13.346 INFO:teuthology.orchestra.run.smithi071.stderr:dumped monmap epoch 1 2024-02-20T12:58:14.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:58:14 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:58:13.343188+0000 mon.smithi064 (mon.0) 238 : audit [DBG] from='client.? 172.21.15.71:0/1463812651' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-20T12:58:15.071 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-20T12:58:15.071 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/daemon-base:latest-pacific shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 680ee126-cfee-11ee-95bb-87774f69a715 -- ceph mon dump -f json 2024-02-20T12:58:15.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:58:15 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:58:13.773246+0000 mgr.smithi064.vriyor (mgr.14156) 228 : cluster [DBG] pgmap v214: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:58:17.057 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:17.055+0000 7f15666c6700 1 -- 172.21.15.71:0/471586199 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f15600f6590 msgr2=0x7f15600f69b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:58:17.057 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:17.055+0000 7f15666c6700 1 --2- 172.21.15.71:0/471586199 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f15600f6590 0x7f15600f69b0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 rx=0x7f1554004760 tx=0x7f1554009ad0).stop 2024-02-20T12:58:17.057 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:17.055+0000 7f15666c6700 1 -- 172.21.15.71:0/471586199 shutdown_connections 2024-02-20T12:58:17.057 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:17.055+0000 7f15666c6700 1 --2- 172.21.15.71:0/471586199 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f15600f6590 0x7f15600f69b0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:58:17.058 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:17.055+0000 7f15666c6700 1 -- 172.21.15.71:0/471586199 >> 172.21.15.71:0/471586199 conn(0x7f15600f1cd0 msgr2=0x7f15600f4110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:58:17.058 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:17.055+0000 7f15666c6700 1 -- 172.21.15.71:0/471586199 shutdown_connections 2024-02-20T12:58:17.058 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:17.055+0000 7f15666c6700 1 -- 172.21.15.71:0/471586199 wait complete. 2024-02-20T12:58:17.058 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:17.055+0000 7f15666c6700 1 Processor -- start 2024-02-20T12:58:17.058 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:17.056+0000 7f15666c6700 1 -- start start 2024-02-20T12:58:17.058 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:17.056+0000 7f15666c6700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f15600f6590 0x7f156006ba80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:58:17.058 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:17.056+0000 7f15666c6700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1560069f00 con 0x7f15600f6590 2024-02-20T12:58:17.058 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:17.056+0000 7f155ffff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f15600f6590 0x7f156006ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:58:17.058 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:17.056+0000 7f155ffff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f15600f6590 0x7f156006ba80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.71:35270/0 (socket says 172.21.15.71:35270) 2024-02-20T12:58:17.058 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:17.056+0000 7f155ffff700 1 -- 172.21.15.71:0/2849529163 learned_addr learned my addr 172.21.15.71:0/2849529163 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-02-20T12:58:17.058 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:17.056+0000 7f155ffff700 1 -- 172.21.15.71:0/2849529163 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1554009d70 con 0x7f15600f6590 2024-02-20T12:58:17.058 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:17.056+0000 7f155ffff700 1 --2- 172.21.15.71:0/2849529163 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f15600f6590 0x7f156006ba80 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 rx=0x7f1554004170 tx=0x7f1554009ad0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:58:17.058 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:17.057+0000 7f155cff9700 1 -- 172.21.15.71:0/2849529163 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f155400e040 con 0x7f15600f6590 2024-02-20T12:58:17.058 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:17.057+0000 7f15666c6700 1 -- 172.21.15.71:0/2849529163 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f156006a0a0 con 0x7f15600f6590 2024-02-20T12:58:17.059 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:17.057+0000 7f155cff9700 1 -- 172.21.15.71:0/2849529163 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(25 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f1554008ef0 con 0x7f15600f6590 2024-02-20T12:58:17.059 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:17.057+0000 7f155cff9700 1 -- 172.21.15.71:0/2849529163 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1554014c90 con 0x7f15600f6590 2024-02-20T12:58:17.059 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:17.057+0000 7f15666c6700 1 -- 172.21.15.71:0/2849529163 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f156006a530 con 0x7f15600f6590 2024-02-20T12:58:17.060 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:17.057+0000 7f155cff9700 1 -- 172.21.15.71:0/2849529163 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 37151+0+0 (secure 0 0 0) 0x7f15540283f0 con 0x7f15600f6590 2024-02-20T12:58:17.060 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:17.058+0000 7f155cff9700 1 --2- 172.21.15.71:0/2849529163 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f1548031090 0x7f1548033550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:58:17.060 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:17.058+0000 7f155cff9700 1 -- 172.21.15.71:0/2849529163 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1065+0+0 (secure 0 0 0) 0x7f1554042230 con 0x7f15600f6590 2024-02-20T12:58:17.060 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:17.058+0000 7f15666c6700 1 -- 172.21.15.71:0/2849529163 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15600591e0 con 0x7f15600f6590 2024-02-20T12:58:17.060 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:17.058+0000 7f155f7fe700 1 --2- 172.21.15.71:0/2849529163 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f1548031090 0x7f1548033550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:58:17.060 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:17.058+0000 7f155f7fe700 1 --2- 172.21.15.71:0/2849529163 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f1548031090 0x7f1548033550 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 rx=0x7f15500096a0 tx=0x7f1550006b40).ready entity=mgr.14156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:58:17.063 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:17.063+0000 7f155cff9700 1 -- 172.21.15.71:0/2849529163 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+153237 (secure 0 0 0) 0x7f15540268f0 con 0x7f15600f6590 2024-02-20T12:58:17.268 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:17.267+0000 7f15666c6700 1 -- 172.21.15.71:0/2849529163 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f15600fc080 con 0x7f15600f6590 2024-02-20T12:58:17.268 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:17.268+0000 7f155cff9700 1 -- 172.21.15.71:0/2849529163 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7f1554026300 con 0x7f15600f6590 2024-02-20T12:58:17.269 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-20T12:58:17.269 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":1,"fsid":"680ee126-cfee-11ee-95bb-87774f69a715","modified":"2024-02-20T12:49:44.699059Z","created":"2024-02-20T12:49:44.699059Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi064","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:3300","nonce":0},{"type":"v1","addr":"172.21.15.64:6789","nonce":0}]},"addr":"172.21.15.64:6789/0","public_addr":"172.21.15.64:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-20T12:58:17.270 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:17.269+0000 7f15666c6700 1 -- 172.21.15.71:0/2849529163 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f1548031090 msgr2=0x7f1548033550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:58:17.270 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:17.269+0000 7f15666c6700 1 --2- 172.21.15.71:0/2849529163 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f1548031090 0x7f1548033550 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 rx=0x7f15500096a0 tx=0x7f1550006b40).stop 2024-02-20T12:58:17.270 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:17.269+0000 7f15666c6700 1 -- 172.21.15.71:0/2849529163 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f15600f6590 msgr2=0x7f156006ba80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:58:17.270 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:17.269+0000 7f15666c6700 1 --2- 172.21.15.71:0/2849529163 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f15600f6590 0x7f156006ba80 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 rx=0x7f1554004170 tx=0x7f1554009ad0).stop 2024-02-20T12:58:17.270 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:17.269+0000 7f15666c6700 1 -- 172.21.15.71:0/2849529163 shutdown_connections 2024-02-20T12:58:17.270 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:17.269+0000 7f15666c6700 1 --2- 172.21.15.71:0/2849529163 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f1548031090 0x7f1548033550 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:58:17.270 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:17.269+0000 7f15666c6700 1 --2- 172.21.15.71:0/2849529163 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f15600f6590 0x7f156006ba80 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:58:17.270 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:17.269+0000 7f15666c6700 1 -- 172.21.15.71:0/2849529163 >> 172.21.15.71:0/2849529163 conn(0x7f15600f1cd0 msgr2=0x7f15600f27c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:58:17.270 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:17.269+0000 7f15666c6700 1 -- 172.21.15.71:0/2849529163 shutdown_connections 2024-02-20T12:58:17.271 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:17.269+0000 7f15666c6700 1 -- 172.21.15.71:0/2849529163 wait complete. 2024-02-20T12:58:17.271 INFO:teuthology.orchestra.run.smithi071.stderr:dumped monmap epoch 1 2024-02-20T12:58:17.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:58:17 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:58:15.773654+0000 mgr.smithi064.vriyor (mgr.14156) 229 : cluster [DBG] pgmap v215: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:58:18.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:58:18 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:58:17.268663+0000 mon.smithi064 (mon.0) 239 : audit [DBG] from='client.? 172.21.15.71:0/2849529163' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-20T12:58:18.937 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-20T12:58:18.937 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/daemon-base:latest-pacific shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 680ee126-cfee-11ee-95bb-87774f69a715 -- ceph mon dump -f json 2024-02-20T12:58:19.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:58:19 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:58:17.774115+0000 mgr.smithi064.vriyor (mgr.14156) 230 : cluster [DBG] pgmap v216: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:58:20.853 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:20.852+0000 7fe7873db700 1 -- 172.21.15.71:0/896605919 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe7800f63e0 msgr2=0x7fe7800f6800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:58:20.853 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:20.852+0000 7fe7873db700 1 --2- 172.21.15.71:0/896605919 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe7800f63e0 0x7fe7800f6800 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 rx=0x7fe77c009fb0 tx=0x7fe77c009b30).stop 2024-02-20T12:58:20.853 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:20.852+0000 7fe7873db700 1 -- 172.21.15.71:0/896605919 shutdown_connections 2024-02-20T12:58:20.853 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:20.852+0000 7fe7873db700 1 --2- 172.21.15.71:0/896605919 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe7800f63e0 0x7fe7800f6800 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:58:20.853 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:20.852+0000 7fe7873db700 1 -- 172.21.15.71:0/896605919 >> 172.21.15.71:0/896605919 conn(0x7fe7800f1b40 msgr2=0x7fe7800f3f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:58:20.853 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:20.852+0000 7fe7873db700 1 -- 172.21.15.71:0/896605919 shutdown_connections 2024-02-20T12:58:20.853 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:20.852+0000 7fe7873db700 1 -- 172.21.15.71:0/896605919 wait complete. 2024-02-20T12:58:20.855 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:20.853+0000 7fe7873db700 1 Processor -- start 2024-02-20T12:58:20.855 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:20.853+0000 7fe7873db700 1 -- start start 2024-02-20T12:58:20.856 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:20.853+0000 7fe7873db700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe7800f63e0 0x7fe780106620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:58:20.856 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:20.853+0000 7fe7873db700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe780106b10 con 0x7fe7800f63e0 2024-02-20T12:58:20.856 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:20.853+0000 7fe7863d9700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe7800f63e0 0x7fe780106620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:58:20.856 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:20.853+0000 7fe7863d9700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe7800f63e0 0x7fe780106620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.71:36542/0 (socket says 172.21.15.71:36542) 2024-02-20T12:58:20.856 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:20.853+0000 7fe7863d9700 1 -- 172.21.15.71:0/1186191683 learned_addr learned my addr 172.21.15.71:0/1186191683 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-02-20T12:58:20.856 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:20.854+0000 7fe7863d9700 1 -- 172.21.15.71:0/1186191683 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe77c005040 con 0x7fe7800f63e0 2024-02-20T12:58:20.856 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:20.854+0000 7fe7863d9700 1 --2- 172.21.15.71:0/1186191683 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe7800f63e0 0x7fe780106620 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 rx=0x7fe77c0089f0 tx=0x7fe77c00b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:58:20.856 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:20.854+0000 7fe776ffd700 1 -- 172.21.15.71:0/1186191683 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe77c009320 con 0x7fe7800f63e0 2024-02-20T12:58:20.856 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:20.854+0000 7fe7873db700 1 -- 172.21.15.71:0/1186191683 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe780106d10 con 0x7fe7800f63e0 2024-02-20T12:58:20.856 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:20.854+0000 7fe776ffd700 1 -- 172.21.15.71:0/1186191683 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(25 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7fe77c013ec0 con 0x7fe7800f63e0 2024-02-20T12:58:20.856 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:20.854+0000 7fe776ffd700 1 -- 172.21.15.71:0/1186191683 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe77c014d30 con 0x7fe7800f63e0 2024-02-20T12:58:20.856 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:20.854+0000 7fe7873db700 1 -- 172.21.15.71:0/1186191683 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7801071a0 con 0x7fe7800f63e0 2024-02-20T12:58:20.856 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:20.855+0000 7fe776ffd700 1 -- 172.21.15.71:0/1186191683 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 37151+0+0 (secure 0 0 0) 0x7fe77c0263a0 con 0x7fe7800f63e0 2024-02-20T12:58:20.857 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:20.855+0000 7fe776ffd700 1 --2- 172.21.15.71:0/1186191683 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fe76c031090 0x7fe76c033550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:58:20.857 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:20.855+0000 7fe776ffd700 1 -- 172.21.15.71:0/1186191683 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1065+0+0 (secure 0 0 0) 0x7fe77c0424a0 con 0x7fe7800f63e0 2024-02-20T12:58:20.857 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:20.855+0000 7fe7873db700 1 -- 172.21.15.71:0/1186191683 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7800591e0 con 0x7fe7800f63e0 2024-02-20T12:58:20.857 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:20.855+0000 7fe785bd8700 1 --2- 172.21.15.71:0/1186191683 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fe76c031090 0x7fe76c033550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:58:20.857 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:20.856+0000 7fe785bd8700 1 --2- 172.21.15.71:0/1186191683 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fe76c031090 0x7fe76c033550 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 rx=0x7fe7700096a0 tx=0x7fe770006b40).ready entity=mgr.14156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:58:20.860 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:20.860+0000 7fe776ffd700 1 -- 172.21.15.71:0/1186191683 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+153237 (secure 0 0 0) 0x7fe77c0266d0 con 0x7fe7800f63e0 2024-02-20T12:58:21.065 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:21.064+0000 7fe7873db700 1 -- 172.21.15.71:0/1186191683 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fe780045bb0 con 0x7fe7800f63e0 2024-02-20T12:58:21.065 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:21.065+0000 7fe776ffd700 1 -- 172.21.15.71:0/1186191683 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7fe77c02b020 con 0x7fe7800f63e0 2024-02-20T12:58:21.065 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-20T12:58:21.065 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":1,"fsid":"680ee126-cfee-11ee-95bb-87774f69a715","modified":"2024-02-20T12:49:44.699059Z","created":"2024-02-20T12:49:44.699059Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi064","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:3300","nonce":0},{"type":"v1","addr":"172.21.15.64:6789","nonce":0}]},"addr":"172.21.15.64:6789/0","public_addr":"172.21.15.64:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-20T12:58:21.067 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:21.066+0000 7fe7873db700 1 -- 172.21.15.71:0/1186191683 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fe76c031090 msgr2=0x7fe76c033550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:58:21.067 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:21.066+0000 7fe7873db700 1 --2- 172.21.15.71:0/1186191683 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fe76c031090 0x7fe76c033550 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 rx=0x7fe7700096a0 tx=0x7fe770006b40).stop 2024-02-20T12:58:21.067 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:21.066+0000 7fe7873db700 1 -- 172.21.15.71:0/1186191683 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe7800f63e0 msgr2=0x7fe780106620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:58:21.067 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:21.066+0000 7fe7873db700 1 --2- 172.21.15.71:0/1186191683 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe7800f63e0 0x7fe780106620 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 rx=0x7fe77c0089f0 tx=0x7fe77c00b040).stop 2024-02-20T12:58:21.067 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:21.066+0000 7fe7873db700 1 -- 172.21.15.71:0/1186191683 shutdown_connections 2024-02-20T12:58:21.067 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:21.066+0000 7fe7873db700 1 --2- 172.21.15.71:0/1186191683 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fe76c031090 0x7fe76c033550 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:58:21.067 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:21.066+0000 7fe7873db700 1 --2- 172.21.15.71:0/1186191683 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe7800f63e0 0x7fe780106620 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:58:21.067 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:21.066+0000 7fe7873db700 1 -- 172.21.15.71:0/1186191683 >> 172.21.15.71:0/1186191683 conn(0x7fe7800f1b40 msgr2=0x7fe7800f2660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:58:21.067 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:21.066+0000 7fe7873db700 1 -- 172.21.15.71:0/1186191683 shutdown_connections 2024-02-20T12:58:21.068 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:21.067+0000 7fe7873db700 1 -- 172.21.15.71:0/1186191683 wait complete. 2024-02-20T12:58:21.068 INFO:teuthology.orchestra.run.smithi071.stderr:dumped monmap epoch 1 2024-02-20T12:58:21.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:58:21 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:58:19.774464+0000 mgr.smithi064.vriyor (mgr.14156) 231 : cluster [DBG] pgmap v217: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:58:22.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:58:22 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:58:21.065772+0000 mon.smithi064 (mon.0) 240 : audit [DBG] from='client.? 172.21.15.71:0/1186191683' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-20T12:58:22.799 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-20T12:58:22.800 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/daemon-base:latest-pacific shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 680ee126-cfee-11ee-95bb-87774f69a715 -- ceph mon dump -f json 2024-02-20T12:58:23.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:58:23 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:58:21.774901+0000 mgr.smithi064.vriyor (mgr.14156) 232 : cluster [DBG] pgmap v218: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:58:24.611 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:24.591+0000 7fb6aeee4700 1 -- 172.21.15.71:0/1099166248 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb6a80f63e0 msgr2=0x7fb6a80f6800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:58:24.612 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:24.591+0000 7fb6aeee4700 1 --2- 172.21.15.71:0/1099166248 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb6a80f63e0 0x7fb6a80f6800 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 rx=0x7fb6a4009fb0 tx=0x7fb6a4009b30).stop 2024-02-20T12:58:24.612 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:24.591+0000 7fb6aeee4700 1 -- 172.21.15.71:0/1099166248 shutdown_connections 2024-02-20T12:58:24.612 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:24.591+0000 7fb6aeee4700 1 --2- 172.21.15.71:0/1099166248 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb6a80f63e0 0x7fb6a80f6800 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:58:24.612 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:24.591+0000 7fb6aeee4700 1 -- 172.21.15.71:0/1099166248 >> 172.21.15.71:0/1099166248 conn(0x7fb6a80f1b40 msgr2=0x7fb6a80f3f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:58:24.612 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:24.592+0000 7fb6aeee4700 1 -- 172.21.15.71:0/1099166248 shutdown_connections 2024-02-20T12:58:24.612 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:24.592+0000 7fb6aeee4700 1 -- 172.21.15.71:0/1099166248 wait complete. 2024-02-20T12:58:24.612 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:24.592+0000 7fb6aeee4700 1 Processor -- start 2024-02-20T12:58:24.612 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:24.592+0000 7fb6aeee4700 1 -- start start 2024-02-20T12:58:24.612 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:24.592+0000 7fb6aeee4700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb6a80f63e0 0x7fb6a8106620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:58:24.612 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:24.592+0000 7fb6aeee4700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6a8106b10 con 0x7fb6a80f63e0 2024-02-20T12:58:24.612 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:24.592+0000 7fb6adee2700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb6a80f63e0 0x7fb6a8106620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:58:24.612 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:24.592+0000 7fb6adee2700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb6a80f63e0 0x7fb6a8106620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.71:36550/0 (socket says 172.21.15.71:36550) 2024-02-20T12:58:24.613 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:24.592+0000 7fb6adee2700 1 -- 172.21.15.71:0/403553392 learned_addr learned my addr 172.21.15.71:0/403553392 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-02-20T12:58:24.613 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:24.593+0000 7fb6adee2700 1 -- 172.21.15.71:0/403553392 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6a4005040 con 0x7fb6a80f63e0 2024-02-20T12:58:24.613 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:24.593+0000 7fb6adee2700 1 --2- 172.21.15.71:0/403553392 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb6a80f63e0 0x7fb6a8106620 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 rx=0x7fb6a40089f0 tx=0x7fb6a400b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:58:24.613 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:24.593+0000 7fb69e7fc700 1 -- 172.21.15.71:0/403553392 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb6a4009320 con 0x7fb6a80f63e0 2024-02-20T12:58:24.613 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:24.593+0000 7fb6aeee4700 1 -- 172.21.15.71:0/403553392 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6a8106d10 con 0x7fb6a80f63e0 2024-02-20T12:58:24.613 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:24.593+0000 7fb69e7fc700 1 -- 172.21.15.71:0/403553392 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(25 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7fb6a4013ec0 con 0x7fb6a80f63e0 2024-02-20T12:58:24.613 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:24.593+0000 7fb69e7fc700 1 -- 172.21.15.71:0/403553392 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb6a4014d30 con 0x7fb6a80f63e0 2024-02-20T12:58:24.613 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:24.593+0000 7fb6aeee4700 1 -- 172.21.15.71:0/403553392 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6a81071a0 con 0x7fb6a80f63e0 2024-02-20T12:58:24.613 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:24.594+0000 7fb69e7fc700 1 -- 172.21.15.71:0/403553392 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 37151+0+0 (secure 0 0 0) 0x7fb6a40263a0 con 0x7fb6a80f63e0 2024-02-20T12:58:24.613 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:24.594+0000 7fb69e7fc700 1 --2- 172.21.15.71:0/403553392 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fb694031090 0x7fb694033550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:58:24.613 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:24.594+0000 7fb69e7fc700 1 -- 172.21.15.71:0/403553392 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1065+0+0 (secure 0 0 0) 0x7fb6a40424c0 con 0x7fb6a80f63e0 2024-02-20T12:58:24.613 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:24.594+0000 7fb6aeee4700 1 -- 172.21.15.71:0/403553392 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6a80591e0 con 0x7fb6a80f63e0 2024-02-20T12:58:24.613 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:24.594+0000 7fb6ad6e1700 1 --2- 172.21.15.71:0/403553392 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fb694031090 0x7fb694033550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:58:24.613 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:24.595+0000 7fb6ad6e1700 1 --2- 172.21.15.71:0/403553392 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fb694031090 0x7fb694033550 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 rx=0x7fb6980096a0 tx=0x7fb698006b40).ready entity=mgr.14156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:58:24.614 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:24.599+0000 7fb69e7fc700 1 -- 172.21.15.71:0/403553392 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+153237 (secure 0 0 0) 0x7fb6a4012920 con 0x7fb6a80f63e0 2024-02-20T12:58:24.810 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:24.808+0000 7fb6aeee4700 1 -- 172.21.15.71:0/403553392 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fb6a8045bb0 con 0x7fb6a80f63e0 2024-02-20T12:58:24.810 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:24.809+0000 7fb69e7fc700 1 -- 172.21.15.71:0/403553392 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7fb6a402b020 con 0x7fb6a80f63e0 2024-02-20T12:58:24.811 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-20T12:58:24.811 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":1,"fsid":"680ee126-cfee-11ee-95bb-87774f69a715","modified":"2024-02-20T12:49:44.699059Z","created":"2024-02-20T12:49:44.699059Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi064","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:3300","nonce":0},{"type":"v1","addr":"172.21.15.64:6789","nonce":0}]},"addr":"172.21.15.64:6789/0","public_addr":"172.21.15.64:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-20T12:58:24.812 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:24.811+0000 7fb6aeee4700 1 -- 172.21.15.71:0/403553392 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fb694031090 msgr2=0x7fb694033550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:58:24.812 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:24.811+0000 7fb6aeee4700 1 --2- 172.21.15.71:0/403553392 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fb694031090 0x7fb694033550 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 rx=0x7fb6980096a0 tx=0x7fb698006b40).stop 2024-02-20T12:58:24.812 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:24.811+0000 7fb6aeee4700 1 -- 172.21.15.71:0/403553392 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb6a80f63e0 msgr2=0x7fb6a8106620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:58:24.812 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:24.811+0000 7fb6aeee4700 1 --2- 172.21.15.71:0/403553392 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb6a80f63e0 0x7fb6a8106620 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 rx=0x7fb6a40089f0 tx=0x7fb6a400b040).stop 2024-02-20T12:58:24.812 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:24.811+0000 7fb6aeee4700 1 -- 172.21.15.71:0/403553392 shutdown_connections 2024-02-20T12:58:24.812 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:24.811+0000 7fb6aeee4700 1 --2- 172.21.15.71:0/403553392 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fb694031090 0x7fb694033550 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:58:24.812 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:24.811+0000 7fb6aeee4700 1 --2- 172.21.15.71:0/403553392 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb6a80f63e0 0x7fb6a8106620 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:58:24.812 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:24.811+0000 7fb6aeee4700 1 -- 172.21.15.71:0/403553392 >> 172.21.15.71:0/403553392 conn(0x7fb6a80f1b40 msgr2=0x7fb6a80f2660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:58:24.813 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:24.811+0000 7fb6aeee4700 1 -- 172.21.15.71:0/403553392 shutdown_connections 2024-02-20T12:58:24.813 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:24.811+0000 7fb6aeee4700 1 -- 172.21.15.71:0/403553392 wait complete. 2024-02-20T12:58:24.813 INFO:teuthology.orchestra.run.smithi071.stderr:dumped monmap epoch 1 2024-02-20T12:58:25.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:58:25 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:58:23.775286+0000 mgr.smithi064.vriyor (mgr.14156) 233 : cluster [DBG] pgmap v219: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:58:25.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:58:25 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:58:24.810914+0000 mon.smithi064 (mon.0) 241 : audit [DBG] from='client.? 172.21.15.71:0/403553392' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-20T12:58:26.607 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-20T12:58:26.608 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/daemon-base:latest-pacific shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 680ee126-cfee-11ee-95bb-87774f69a715 -- ceph mon dump -f json 2024-02-20T12:58:27.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:58:27 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:58:25.775695+0000 mgr.smithi064.vriyor (mgr.14156) 234 : cluster [DBG] pgmap v220: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:58:28.729 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:28.726+0000 7ffa24438700 1 -- 172.21.15.71:0/1693283611 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffa1c0f5480 msgr2=0x7ffa1c0f58a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:58:28.729 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:28.726+0000 7ffa24438700 1 --2- 172.21.15.71:0/1693283611 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffa1c0f5480 0x7ffa1c0f58a0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 rx=0x7ffa0c009fb0 tx=0x7ffa0c009b30).stop 2024-02-20T12:58:28.729 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:28.726+0000 7ffa24438700 1 -- 172.21.15.71:0/1693283611 shutdown_connections 2024-02-20T12:58:28.729 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:28.726+0000 7ffa24438700 1 --2- 172.21.15.71:0/1693283611 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffa1c0f5480 0x7ffa1c0f58a0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:58:28.729 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:28.726+0000 7ffa24438700 1 -- 172.21.15.71:0/1693283611 >> 172.21.15.71:0/1693283611 conn(0x7ffa1c0f0be0 msgr2=0x7ffa1c0f3000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:58:28.729 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:28.726+0000 7ffa24438700 1 -- 172.21.15.71:0/1693283611 shutdown_connections 2024-02-20T12:58:28.729 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:28.726+0000 7ffa24438700 1 -- 172.21.15.71:0/1693283611 wait complete. 2024-02-20T12:58:28.729 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:28.727+0000 7ffa24438700 1 Processor -- start 2024-02-20T12:58:28.730 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:28.727+0000 7ffa24438700 1 -- start start 2024-02-20T12:58:28.730 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:28.727+0000 7ffa24438700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffa1c0f5480 0x7ffa1c10b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:58:28.730 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:28.727+0000 7ffa24438700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa1c107300 con 0x7ffa1c0f5480 2024-02-20T12:58:28.730 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:28.727+0000 7ffa221d4700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffa1c0f5480 0x7ffa1c10b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:58:28.730 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:28.727+0000 7ffa221d4700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffa1c0f5480 0x7ffa1c10b910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.71:48184/0 (socket says 172.21.15.71:48184) 2024-02-20T12:58:28.730 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:28.727+0000 7ffa221d4700 1 -- 172.21.15.71:0/4179032534 learned_addr learned my addr 172.21.15.71:0/4179032534 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-02-20T12:58:28.730 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:28.728+0000 7ffa221d4700 1 -- 172.21.15.71:0/4179032534 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa0c005040 con 0x7ffa1c0f5480 2024-02-20T12:58:28.730 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:28.728+0000 7ffa221d4700 1 --2- 172.21.15.71:0/4179032534 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffa1c0f5480 0x7ffa1c10b910 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 rx=0x7ffa0c0089f0 tx=0x7ffa0c00b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:58:28.731 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:28.728+0000 7ffa12ffd700 1 -- 172.21.15.71:0/4179032534 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ffa0c009320 con 0x7ffa1c0f5480 2024-02-20T12:58:28.731 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:28.728+0000 7ffa24438700 1 -- 172.21.15.71:0/4179032534 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa1c060300 con 0x7ffa1c0f5480 2024-02-20T12:58:28.732 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:28.728+0000 7ffa24438700 1 -- 172.21.15.71:0/4179032534 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa1c10be00 con 0x7ffa1c0f5480 2024-02-20T12:58:28.732 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:28.728+0000 7ffa12ffd700 1 -- 172.21.15.71:0/4179032534 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(25 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7ffa0c013ec0 con 0x7ffa1c0f5480 2024-02-20T12:58:28.732 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:28.728+0000 7ffa12ffd700 1 -- 172.21.15.71:0/4179032534 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ffa0c014d20 con 0x7ffa1c0f5480 2024-02-20T12:58:28.732 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:28.729+0000 7ffa12ffd700 1 -- 172.21.15.71:0/4179032534 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 37151+0+0 (secure 0 0 0) 0x7ffa0c014420 con 0x7ffa1c0f5480 2024-02-20T12:58:28.732 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:28.729+0000 7ffa24438700 1 -- 172.21.15.71:0/4179032534 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa1c045bb0 con 0x7ffa1c0f5480 2024-02-20T12:58:28.732 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:28.729+0000 7ffa12ffd700 1 --2- 172.21.15.71:0/4179032534 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7ffa08031090 0x7ffa08033550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:58:28.732 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:28.729+0000 7ffa12ffd700 1 -- 172.21.15.71:0/4179032534 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1065+0+0 (secure 0 0 0) 0x7ffa0c042470 con 0x7ffa1c0f5480 2024-02-20T12:58:28.732 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:28.729+0000 7ffa219d3700 1 --2- 172.21.15.71:0/4179032534 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7ffa08031090 0x7ffa08033550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:58:28.732 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:28.730+0000 7ffa219d3700 1 --2- 172.21.15.71:0/4179032534 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7ffa08031090 0x7ffa08033550 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 rx=0x7ffa180096a0 tx=0x7ffa18006b40).ready entity=mgr.14156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:58:28.736 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:28.734+0000 7ffa12ffd700 1 -- 172.21.15.71:0/4179032534 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+153237 (secure 0 0 0) 0x7ffa0c01db70 con 0x7ffa1c0f5480 2024-02-20T12:58:28.947 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:28.946+0000 7ffa24438700 1 -- 172.21.15.71:0/4179032534 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7ffa1c0faf70 con 0x7ffa1c0f5480 2024-02-20T12:58:28.948 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:28.946+0000 7ffa12ffd700 1 -- 172.21.15.71:0/4179032534 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7ffa0c01db70 con 0x7ffa1c0f5480 2024-02-20T12:58:28.948 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-20T12:58:28.948 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":1,"fsid":"680ee126-cfee-11ee-95bb-87774f69a715","modified":"2024-02-20T12:49:44.699059Z","created":"2024-02-20T12:49:44.699059Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi064","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:3300","nonce":0},{"type":"v1","addr":"172.21.15.64:6789","nonce":0}]},"addr":"172.21.15.64:6789/0","public_addr":"172.21.15.64:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-20T12:58:28.950 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:28.948+0000 7ffa24438700 1 -- 172.21.15.71:0/4179032534 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7ffa08031090 msgr2=0x7ffa08033550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:58:28.950 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:28.948+0000 7ffa24438700 1 --2- 172.21.15.71:0/4179032534 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7ffa08031090 0x7ffa08033550 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 rx=0x7ffa180096a0 tx=0x7ffa18006b40).stop 2024-02-20T12:58:28.950 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:28.948+0000 7ffa24438700 1 -- 172.21.15.71:0/4179032534 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffa1c0f5480 msgr2=0x7ffa1c10b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:58:28.950 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:28.948+0000 7ffa24438700 1 --2- 172.21.15.71:0/4179032534 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffa1c0f5480 0x7ffa1c10b910 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 rx=0x7ffa0c0089f0 tx=0x7ffa0c00b040).stop 2024-02-20T12:58:28.950 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:28.948+0000 7ffa24438700 1 -- 172.21.15.71:0/4179032534 shutdown_connections 2024-02-20T12:58:28.950 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:28.948+0000 7ffa24438700 1 --2- 172.21.15.71:0/4179032534 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7ffa08031090 0x7ffa08033550 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:58:28.950 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:28.948+0000 7ffa24438700 1 --2- 172.21.15.71:0/4179032534 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffa1c0f5480 0x7ffa1c10b910 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:58:28.950 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:28.948+0000 7ffa24438700 1 -- 172.21.15.71:0/4179032534 >> 172.21.15.71:0/4179032534 conn(0x7ffa1c0f0be0 msgr2=0x7ffa1c0f16d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:58:28.951 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:28.948+0000 7ffa24438700 1 -- 172.21.15.71:0/4179032534 shutdown_connections 2024-02-20T12:58:28.951 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:28.948+0000 7ffa24438700 1 -- 172.21.15.71:0/4179032534 wait complete. 2024-02-20T12:58:28.951 INFO:teuthology.orchestra.run.smithi071.stderr:dumped monmap epoch 1 2024-02-20T12:58:29.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:58:29 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:58:27.776143+0000 mgr.smithi064.vriyor (mgr.14156) 235 : cluster [DBG] pgmap v221: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:58:29.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:58:29 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:58:28.948347+0000 mon.smithi064 (mon.0) 242 : audit [DBG] from='client.? 172.21.15.71:0/4179032534' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-20T12:58:30.696 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-20T12:58:30.696 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/daemon-base:latest-pacific shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 680ee126-cfee-11ee-95bb-87774f69a715 -- ceph mon dump -f json 2024-02-20T12:58:31.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:58:31 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:58:29.776488+0000 mgr.smithi064.vriyor (mgr.14156) 236 : cluster [DBG] pgmap v222: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:58:32.733 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:32.731+0000 7fbfb2f66700 1 -- 172.21.15.71:0/1287993136 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbfac0f6590 msgr2=0x7fbfac0f69b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:58:32.733 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:32.731+0000 7fbfb2f66700 1 --2- 172.21.15.71:0/1287993136 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbfac0f6590 0x7fbfac0f69b0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 rx=0x7fbfa0004760 tx=0x7fbfa0009ad0).stop 2024-02-20T12:58:32.733 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:32.731+0000 7fbfb2f66700 1 -- 172.21.15.71:0/1287993136 shutdown_connections 2024-02-20T12:58:32.734 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:32.731+0000 7fbfb2f66700 1 --2- 172.21.15.71:0/1287993136 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbfac0f6590 0x7fbfac0f69b0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:58:32.734 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:32.731+0000 7fbfb2f66700 1 -- 172.21.15.71:0/1287993136 >> 172.21.15.71:0/1287993136 conn(0x7fbfac0f1cd0 msgr2=0x7fbfac0f4110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:58:32.734 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:32.731+0000 7fbfb2f66700 1 -- 172.21.15.71:0/1287993136 shutdown_connections 2024-02-20T12:58:32.734 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:32.731+0000 7fbfb2f66700 1 -- 172.21.15.71:0/1287993136 wait complete. 2024-02-20T12:58:32.734 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:32.732+0000 7fbfb2f66700 1 Processor -- start 2024-02-20T12:58:32.734 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:32.732+0000 7fbfb2f66700 1 -- start start 2024-02-20T12:58:32.736 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:32.732+0000 7fbfb2f66700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbfac0f6590 0x7fbfac1067c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:58:32.736 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:32.732+0000 7fbfb2f66700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfac106cb0 con 0x7fbfac0f6590 2024-02-20T12:58:32.736 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:32.733+0000 7fbfb0d02700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbfac0f6590 0x7fbfac1067c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:58:32.736 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:32.733+0000 7fbfb0d02700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbfac0f6590 0x7fbfac1067c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.71:48192/0 (socket says 172.21.15.71:48192) 2024-02-20T12:58:32.736 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:32.733+0000 7fbfb0d02700 1 -- 172.21.15.71:0/26020522 learned_addr learned my addr 172.21.15.71:0/26020522 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-02-20T12:58:32.736 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:32.733+0000 7fbfb0d02700 1 -- 172.21.15.71:0/26020522 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfa0009d70 con 0x7fbfac0f6590 2024-02-20T12:58:32.736 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:32.733+0000 7fbfb0d02700 1 --2- 172.21.15.71:0/26020522 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbfac0f6590 0x7fbfac1067c0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 rx=0x7fbfa0004170 tx=0x7fbfa0009ad0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:58:32.736 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:32.733+0000 7fbfa97fa700 1 -- 172.21.15.71:0/26020522 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fbfa000e040 con 0x7fbfac0f6590 2024-02-20T12:58:32.736 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:32.733+0000 7fbfa97fa700 1 -- 172.21.15.71:0/26020522 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(25 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7fbfa0008ef0 con 0x7fbfac0f6590 2024-02-20T12:58:32.736 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:32.733+0000 7fbfb2f66700 1 -- 172.21.15.71:0/26020522 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfac106eb0 con 0x7fbfac0f6590 2024-02-20T12:58:32.736 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:32.733+0000 7fbfa97fa700 1 -- 172.21.15.71:0/26020522 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fbfa0014c90 con 0x7fbfac0f6590 2024-02-20T12:58:32.736 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:32.733+0000 7fbfb2f66700 1 -- 172.21.15.71:0/26020522 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfac107340 con 0x7fbfac0f6590 2024-02-20T12:58:32.737 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:32.734+0000 7fbfa97fa700 1 -- 172.21.15.71:0/26020522 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 37151+0+0 (secure 0 0 0) 0x7fbfa0014420 con 0x7fbfac0f6590 2024-02-20T12:58:32.737 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:32.734+0000 7fbfb2f66700 1 -- 172.21.15.71:0/26020522 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbfac045bb0 con 0x7fbfac0f6590 2024-02-20T12:58:32.737 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:32.734+0000 7fbfa97fa700 1 --2- 172.21.15.71:0/26020522 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fbf9c031090 0x7fbf9c033550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:58:32.737 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:32.734+0000 7fbfa97fa700 1 -- 172.21.15.71:0/26020522 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1065+0+0 (secure 0 0 0) 0x7fbfa00421f0 con 0x7fbfac0f6590 2024-02-20T12:58:32.737 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:32.734+0000 7fbfabfff700 1 --2- 172.21.15.71:0/26020522 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fbf9c031090 0x7fbf9c033550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:58:32.737 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:32.735+0000 7fbfabfff700 1 --2- 172.21.15.71:0/26020522 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fbf9c031090 0x7fbf9c033550 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 rx=0x7fbf980096a0 tx=0x7fbf98006b40).ready entity=mgr.14156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:58:32.740 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:32.739+0000 7fbfa97fa700 1 -- 172.21.15.71:0/26020522 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+153237 (secure 0 0 0) 0x7fbfa0026300 con 0x7fbfac0f6590 2024-02-20T12:58:32.950 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:32.949+0000 7fbfb2f66700 1 -- 172.21.15.71:0/26020522 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fbfac1081f0 con 0x7fbfac0f6590 2024-02-20T12:58:32.951 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:32.949+0000 7fbfa97fa700 1 -- 172.21.15.71:0/26020522 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7fbfa0014e60 con 0x7fbfac0f6590 2024-02-20T12:58:32.951 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-20T12:58:32.951 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":1,"fsid":"680ee126-cfee-11ee-95bb-87774f69a715","modified":"2024-02-20T12:49:44.699059Z","created":"2024-02-20T12:49:44.699059Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi064","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:3300","nonce":0},{"type":"v1","addr":"172.21.15.64:6789","nonce":0}]},"addr":"172.21.15.64:6789/0","public_addr":"172.21.15.64:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-20T12:58:32.952 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:32.951+0000 7fbfb2f66700 1 -- 172.21.15.71:0/26020522 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fbf9c031090 msgr2=0x7fbf9c033550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:58:32.953 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:32.951+0000 7fbfb2f66700 1 --2- 172.21.15.71:0/26020522 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fbf9c031090 0x7fbf9c033550 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 rx=0x7fbf980096a0 tx=0x7fbf98006b40).stop 2024-02-20T12:58:32.953 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:32.951+0000 7fbfb2f66700 1 -- 172.21.15.71:0/26020522 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbfac0f6590 msgr2=0x7fbfac1067c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:58:32.953 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:32.951+0000 7fbfb2f66700 1 --2- 172.21.15.71:0/26020522 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbfac0f6590 0x7fbfac1067c0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 rx=0x7fbfa0004170 tx=0x7fbfa0009ad0).stop 2024-02-20T12:58:32.953 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:32.951+0000 7fbfb2f66700 1 -- 172.21.15.71:0/26020522 shutdown_connections 2024-02-20T12:58:32.953 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:32.951+0000 7fbfb2f66700 1 --2- 172.21.15.71:0/26020522 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fbf9c031090 0x7fbf9c033550 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:58:32.953 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:32.951+0000 7fbfb2f66700 1 --2- 172.21.15.71:0/26020522 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbfac0f6590 0x7fbfac1067c0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:58:32.953 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:32.951+0000 7fbfb2f66700 1 -- 172.21.15.71:0/26020522 >> 172.21.15.71:0/26020522 conn(0x7fbfac0f1cd0 msgr2=0x7fbfac0f27c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:58:32.953 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:32.951+0000 7fbfb2f66700 1 -- 172.21.15.71:0/26020522 shutdown_connections 2024-02-20T12:58:32.953 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:32.951+0000 7fbfb2f66700 1 -- 172.21.15.71:0/26020522 wait complete. 2024-02-20T12:58:32.954 INFO:teuthology.orchestra.run.smithi071.stderr:dumped monmap epoch 1 2024-02-20T12:58:33.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:58:33 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:58:31.776857+0000 mgr.smithi064.vriyor (mgr.14156) 237 : cluster [DBG] pgmap v223: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:58:33.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:58:33 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:58:32.951128+0000 mon.smithi064 (mon.0) 243 : audit [DBG] from='client.? 172.21.15.71:0/26020522' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-20T12:58:34.713 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-20T12:58:34.714 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/daemon-base:latest-pacific shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 680ee126-cfee-11ee-95bb-87774f69a715 -- ceph mon dump -f json 2024-02-20T12:58:35.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:58:35 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:58:33.777143+0000 mgr.smithi064.vriyor (mgr.14156) 238 : cluster [DBG] pgmap v224: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:58:36.622 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:36.619+0000 7f0e9a3e8700 1 -- 172.21.15.71:0/3583179911 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0e940f5480 msgr2=0x7f0e940f58a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:58:36.622 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:36.619+0000 7f0e9a3e8700 1 --2- 172.21.15.71:0/3583179911 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0e940f5480 0x7f0e940f58a0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 rx=0x7f0e84009fb0 tx=0x7f0e84009b30).stop 2024-02-20T12:58:36.622 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:36.619+0000 7f0e9a3e8700 1 -- 172.21.15.71:0/3583179911 shutdown_connections 2024-02-20T12:58:36.622 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:36.619+0000 7f0e9a3e8700 1 --2- 172.21.15.71:0/3583179911 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0e940f5480 0x7f0e940f58a0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:58:36.623 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:36.619+0000 7f0e9a3e8700 1 -- 172.21.15.71:0/3583179911 >> 172.21.15.71:0/3583179911 conn(0x7f0e940f0be0 msgr2=0x7f0e940f3000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:58:36.623 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:36.619+0000 7f0e9a3e8700 1 -- 172.21.15.71:0/3583179911 shutdown_connections 2024-02-20T12:58:36.623 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:36.619+0000 7f0e9a3e8700 1 -- 172.21.15.71:0/3583179911 wait complete. 2024-02-20T12:58:36.623 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:36.619+0000 7f0e9a3e8700 1 Processor -- start 2024-02-20T12:58:36.623 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:36.620+0000 7f0e9a3e8700 1 -- start start 2024-02-20T12:58:36.623 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:36.620+0000 7f0e9a3e8700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0e940f5480 0x7f0e941067e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:58:36.623 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:36.620+0000 7f0e9a3e8700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e94106cd0 con 0x7f0e940f5480 2024-02-20T12:58:36.623 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:36.620+0000 7f0e93fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0e940f5480 0x7f0e941067e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:58:36.623 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:36.620+0000 7f0e93fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0e940f5480 0x7f0e941067e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.71:48210/0 (socket says 172.21.15.71:48210) 2024-02-20T12:58:36.623 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:36.620+0000 7f0e93fff700 1 -- 172.21.15.71:0/4009587161 learned_addr learned my addr 172.21.15.71:0/4009587161 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-02-20T12:58:36.623 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:36.620+0000 7f0e93fff700 1 -- 172.21.15.71:0/4009587161 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e84005040 con 0x7f0e940f5480 2024-02-20T12:58:36.624 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:36.620+0000 7f0e93fff700 1 --2- 172.21.15.71:0/4009587161 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0e940f5480 0x7f0e941067e0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 rx=0x7f0e840089f0 tx=0x7f0e8400b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:58:36.624 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:36.620+0000 7f0e90ff9700 1 -- 172.21.15.71:0/4009587161 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0e84009320 con 0x7f0e940f5480 2024-02-20T12:58:36.624 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:36.621+0000 7f0e9a3e8700 1 -- 172.21.15.71:0/4009587161 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e94106ed0 con 0x7f0e940f5480 2024-02-20T12:58:36.624 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:36.621+0000 7f0e90ff9700 1 -- 172.21.15.71:0/4009587161 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(25 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f0e84013ec0 con 0x7f0e940f5480 2024-02-20T12:58:36.624 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:36.621+0000 7f0e90ff9700 1 -- 172.21.15.71:0/4009587161 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0e84014d30 con 0x7f0e940f5480 2024-02-20T12:58:36.624 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:36.621+0000 7f0e9a3e8700 1 -- 172.21.15.71:0/4009587161 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e94107360 con 0x7f0e940f5480 2024-02-20T12:58:36.625 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:36.621+0000 7f0e90ff9700 1 -- 172.21.15.71:0/4009587161 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 37151+0+0 (secure 0 0 0) 0x7f0e84014420 con 0x7f0e940f5480 2024-02-20T12:58:36.625 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:36.621+0000 7f0e9a3e8700 1 -- 172.21.15.71:0/4009587161 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e94045bb0 con 0x7f0e940f5480 2024-02-20T12:58:36.625 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:36.622+0000 7f0e90ff9700 1 --2- 172.21.15.71:0/4009587161 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f0e7c031090 0x7f0e7c033550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:58:36.625 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:36.622+0000 7f0e90ff9700 1 -- 172.21.15.71:0/4009587161 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1065+0+0 (secure 0 0 0) 0x7f0e840424a0 con 0x7f0e940f5480 2024-02-20T12:58:36.625 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:36.622+0000 7f0e937fe700 1 --2- 172.21.15.71:0/4009587161 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f0e7c031090 0x7f0e7c033550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:58:36.625 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:36.622+0000 7f0e937fe700 1 --2- 172.21.15.71:0/4009587161 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f0e7c031090 0x7f0e7c033550 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 rx=0x7f0e880096a0 tx=0x7f0e88006b40).ready entity=mgr.14156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:58:36.628 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:36.626+0000 7f0e90ff9700 1 -- 172.21.15.71:0/4009587161 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+153237 (secure 0 0 0) 0x7f0e840268c0 con 0x7f0e940f5480 2024-02-20T12:58:36.836 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:36.834+0000 7f0e9a3e8700 1 -- 172.21.15.71:0/4009587161 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f0e94108210 con 0x7f0e940f5480 2024-02-20T12:58:36.836 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:36.835+0000 7f0e90ff9700 1 -- 172.21.15.71:0/4009587161 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7f0e8401d7f0 con 0x7f0e940f5480 2024-02-20T12:58:36.837 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-20T12:58:36.837 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":1,"fsid":"680ee126-cfee-11ee-95bb-87774f69a715","modified":"2024-02-20T12:49:44.699059Z","created":"2024-02-20T12:49:44.699059Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi064","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:3300","nonce":0},{"type":"v1","addr":"172.21.15.64:6789","nonce":0}]},"addr":"172.21.15.64:6789/0","public_addr":"172.21.15.64:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-20T12:58:36.838 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:36.836+0000 7f0e9a3e8700 1 -- 172.21.15.71:0/4009587161 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f0e7c031090 msgr2=0x7f0e7c033550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:58:36.838 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:36.836+0000 7f0e9a3e8700 1 --2- 172.21.15.71:0/4009587161 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f0e7c031090 0x7f0e7c033550 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 rx=0x7f0e880096a0 tx=0x7f0e88006b40).stop 2024-02-20T12:58:36.838 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:36.836+0000 7f0e9a3e8700 1 -- 172.21.15.71:0/4009587161 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0e940f5480 msgr2=0x7f0e941067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:58:36.838 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:36.836+0000 7f0e9a3e8700 1 --2- 172.21.15.71:0/4009587161 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0e940f5480 0x7f0e941067e0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 rx=0x7f0e840089f0 tx=0x7f0e8400b040).stop 2024-02-20T12:58:36.838 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:36.836+0000 7f0e9a3e8700 1 -- 172.21.15.71:0/4009587161 shutdown_connections 2024-02-20T12:58:36.838 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:36.836+0000 7f0e9a3e8700 1 --2- 172.21.15.71:0/4009587161 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f0e7c031090 0x7f0e7c033550 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:58:36.838 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:36.836+0000 7f0e9a3e8700 1 --2- 172.21.15.71:0/4009587161 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0e940f5480 0x7f0e941067e0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:58:36.838 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:36.836+0000 7f0e9a3e8700 1 -- 172.21.15.71:0/4009587161 >> 172.21.15.71:0/4009587161 conn(0x7f0e940f0be0 msgr2=0x7f0e9405ff40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:58:36.838 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:36.836+0000 7f0e9a3e8700 1 -- 172.21.15.71:0/4009587161 shutdown_connections 2024-02-20T12:58:36.838 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:36.836+0000 7f0e9a3e8700 1 -- 172.21.15.71:0/4009587161 wait complete. 2024-02-20T12:58:36.839 INFO:teuthology.orchestra.run.smithi071.stderr:dumped monmap epoch 1 2024-02-20T12:58:37.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:58:37 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:58:35.777596+0000 mgr.smithi064.vriyor (mgr.14156) 239 : cluster [DBG] pgmap v225: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:58:37.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:58:37 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:58:36.836538+0000 mon.smithi064 (mon.0) 244 : audit [DBG] from='client.? 172.21.15.71:0/4009587161' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-20T12:58:38.601 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-20T12:58:38.601 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/daemon-base:latest-pacific shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 680ee126-cfee-11ee-95bb-87774f69a715 -- ceph mon dump -f json 2024-02-20T12:58:39.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:58:39 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:58:37.778092+0000 mgr.smithi064.vriyor (mgr.14156) 240 : cluster [DBG] pgmap v226: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:58:40.487 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:40.485+0000 7ffb1fd82700 1 -- 172.21.15.71:0/883632988 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffb180f54b0 msgr2=0x7ffb180f58d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:58:40.487 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:40.485+0000 7ffb1fd82700 1 --2- 172.21.15.71:0/883632988 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffb180f54b0 0x7ffb180f58d0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 rx=0x7ffb14009fb0 tx=0x7ffb14009b30).stop 2024-02-20T12:58:40.488 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:40.485+0000 7ffb1fd82700 1 -- 172.21.15.71:0/883632988 shutdown_connections 2024-02-20T12:58:40.488 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:40.485+0000 7ffb1fd82700 1 --2- 172.21.15.71:0/883632988 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffb180f54b0 0x7ffb180f58d0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:58:40.488 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:40.485+0000 7ffb1fd82700 1 -- 172.21.15.71:0/883632988 >> 172.21.15.71:0/883632988 conn(0x7ffb180f0c10 msgr2=0x7ffb180f3030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:58:40.488 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:40.485+0000 7ffb1fd82700 1 -- 172.21.15.71:0/883632988 shutdown_connections 2024-02-20T12:58:40.488 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:40.485+0000 7ffb1fd82700 1 -- 172.21.15.71:0/883632988 wait complete. 2024-02-20T12:58:40.488 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:40.485+0000 7ffb1fd82700 1 Processor -- start 2024-02-20T12:58:40.488 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:40.485+0000 7ffb1fd82700 1 -- start start 2024-02-20T12:58:40.488 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:40.485+0000 7ffb1fd82700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffb180f54b0 0x7ffb18109520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:58:40.488 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:40.485+0000 7ffb1fd82700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb18060300 con 0x7ffb180f54b0 2024-02-20T12:58:40.488 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:40.486+0000 7ffb1db1e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffb180f54b0 0x7ffb18109520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:58:40.488 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:40.486+0000 7ffb1db1e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffb180f54b0 0x7ffb18109520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.71:51664/0 (socket says 172.21.15.71:51664) 2024-02-20T12:58:40.489 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:40.486+0000 7ffb1db1e700 1 -- 172.21.15.71:0/2193494397 learned_addr learned my addr 172.21.15.71:0/2193494397 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-02-20T12:58:40.489 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:40.486+0000 7ffb1db1e700 1 -- 172.21.15.71:0/2193494397 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb14005040 con 0x7ffb180f54b0 2024-02-20T12:58:40.489 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:40.486+0000 7ffb1db1e700 1 --2- 172.21.15.71:0/2193494397 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffb180f54b0 0x7ffb18109520 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 rx=0x7ffb140089f0 tx=0x7ffb1400b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:58:40.489 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:40.486+0000 7ffb0e7fc700 1 -- 172.21.15.71:0/2193494397 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ffb14009320 con 0x7ffb180f54b0 2024-02-20T12:58:40.490 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:40.486+0000 7ffb1fd82700 1 -- 172.21.15.71:0/2193494397 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb18060500 con 0x7ffb180f54b0 2024-02-20T12:58:40.491 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:40.486+0000 7ffb0e7fc700 1 -- 172.21.15.71:0/2193494397 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(25 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7ffb14013ec0 con 0x7ffb180f54b0 2024-02-20T12:58:40.491 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:40.486+0000 7ffb0e7fc700 1 -- 172.21.15.71:0/2193494397 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ffb14014d30 con 0x7ffb180f54b0 2024-02-20T12:58:40.491 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:40.486+0000 7ffb1fd82700 1 -- 172.21.15.71:0/2193494397 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb18109bd0 con 0x7ffb180f54b0 2024-02-20T12:58:40.491 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:40.487+0000 7ffb0e7fc700 1 -- 172.21.15.71:0/2193494397 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 37151+0+0 (secure 0 0 0) 0x7ffb140095f0 con 0x7ffb180f54b0 2024-02-20T12:58:40.491 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:40.487+0000 7ffb1fd82700 1 -- 172.21.15.71:0/2193494397 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb180591e0 con 0x7ffb180f54b0 2024-02-20T12:58:40.491 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:40.487+0000 7ffb0e7fc700 1 --2- 172.21.15.71:0/2193494397 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7ffb04031090 0x7ffb04033550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:58:40.491 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:40.487+0000 7ffb0e7fc700 1 -- 172.21.15.71:0/2193494397 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1065+0+0 (secure 0 0 0) 0x7ffb140424a0 con 0x7ffb180f54b0 2024-02-20T12:58:40.492 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:40.487+0000 7ffb1d31d700 1 --2- 172.21.15.71:0/2193494397 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7ffb04031090 0x7ffb04033550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:58:40.492 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:40.488+0000 7ffb1d31d700 1 --2- 172.21.15.71:0/2193494397 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7ffb04031090 0x7ffb04033550 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 rx=0x7ffb080096a0 tx=0x7ffb08006b40).ready entity=mgr.14156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:58:40.494 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:40.492+0000 7ffb0e7fc700 1 -- 172.21.15.71:0/2193494397 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+153237 (secure 0 0 0) 0x7ffb1401d980 con 0x7ffb180f54b0 2024-02-20T12:58:40.700 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:40.699+0000 7ffb1fd82700 1 -- 172.21.15.71:0/2193494397 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7ffb18045bb0 con 0x7ffb180f54b0 2024-02-20T12:58:40.701 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:40.699+0000 7ffb0e7fc700 1 -- 172.21.15.71:0/2193494397 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7ffb1401d380 con 0x7ffb180f54b0 2024-02-20T12:58:40.701 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-20T12:58:40.701 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":1,"fsid":"680ee126-cfee-11ee-95bb-87774f69a715","modified":"2024-02-20T12:49:44.699059Z","created":"2024-02-20T12:49:44.699059Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi064","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:3300","nonce":0},{"type":"v1","addr":"172.21.15.64:6789","nonce":0}]},"addr":"172.21.15.64:6789/0","public_addr":"172.21.15.64:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-20T12:58:40.703 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:40.701+0000 7ffb1fd82700 1 -- 172.21.15.71:0/2193494397 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7ffb04031090 msgr2=0x7ffb04033550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:58:40.703 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:40.701+0000 7ffb1fd82700 1 --2- 172.21.15.71:0/2193494397 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7ffb04031090 0x7ffb04033550 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 rx=0x7ffb080096a0 tx=0x7ffb08006b40).stop 2024-02-20T12:58:40.703 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:40.701+0000 7ffb1fd82700 1 -- 172.21.15.71:0/2193494397 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffb180f54b0 msgr2=0x7ffb18109520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:58:40.703 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:40.701+0000 7ffb1fd82700 1 --2- 172.21.15.71:0/2193494397 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffb180f54b0 0x7ffb18109520 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 rx=0x7ffb140089f0 tx=0x7ffb1400b040).stop 2024-02-20T12:58:40.703 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:40.701+0000 7ffb1fd82700 1 -- 172.21.15.71:0/2193494397 shutdown_connections 2024-02-20T12:58:40.703 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:40.701+0000 7ffb1fd82700 1 --2- 172.21.15.71:0/2193494397 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7ffb04031090 0x7ffb04033550 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:58:40.703 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:40.701+0000 7ffb1fd82700 1 --2- 172.21.15.71:0/2193494397 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffb180f54b0 0x7ffb18109520 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:58:40.703 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:40.701+0000 7ffb1fd82700 1 -- 172.21.15.71:0/2193494397 >> 172.21.15.71:0/2193494397 conn(0x7ffb180f0c10 msgr2=0x7ffb180f1700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:58:40.703 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:40.701+0000 7ffb1fd82700 1 -- 172.21.15.71:0/2193494397 shutdown_connections 2024-02-20T12:58:40.703 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:40.701+0000 7ffb1fd82700 1 -- 172.21.15.71:0/2193494397 wait complete. 2024-02-20T12:58:40.704 INFO:teuthology.orchestra.run.smithi071.stderr:dumped monmap epoch 1 2024-02-20T12:58:41.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:58:41 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:58:39.778371+0000 mgr.smithi064.vriyor (mgr.14156) 241 : cluster [DBG] pgmap v227: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:58:41.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:58:41 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:58:40.701406+0000 mon.smithi064 (mon.0) 245 : audit [DBG] from='client.? 172.21.15.71:0/2193494397' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-20T12:58:42.360 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-20T12:58:42.361 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/daemon-base:latest-pacific shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 680ee126-cfee-11ee-95bb-87774f69a715 -- ceph mon dump -f json 2024-02-20T12:58:43.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:58:43 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:58:41.778726+0000 mgr.smithi064.vriyor (mgr.14156) 242 : cluster [DBG] pgmap v228: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:58:44.374 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:44.355+0000 7f0ed3d7e700 1 -- 172.21.15.71:0/3558698408 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0ecc0602c0 msgr2=0x7f0ecc0f9200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:58:44.374 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:44.355+0000 7f0ed3d7e700 1 --2- 172.21.15.71:0/3558698408 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0ecc0602c0 0x7f0ecc0f9200 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 rx=0x7f0ebc009fb0 tx=0x7f0ebc009b30).stop 2024-02-20T12:58:44.374 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:44.355+0000 7f0ed3d7e700 1 -- 172.21.15.71:0/3558698408 shutdown_connections 2024-02-20T12:58:44.374 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:44.355+0000 7f0ed3d7e700 1 --2- 172.21.15.71:0/3558698408 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0ecc0602c0 0x7f0ecc0f9200 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:58:44.374 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:44.355+0000 7f0ed3d7e700 1 -- 172.21.15.71:0/3558698408 >> 172.21.15.71:0/3558698408 conn(0x7f0ecc0f0be0 msgr2=0x7f0ecc0f3000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:58:44.375 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:44.355+0000 7f0ed3d7e700 1 -- 172.21.15.71:0/3558698408 shutdown_connections 2024-02-20T12:58:44.375 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:44.355+0000 7f0ed3d7e700 1 -- 172.21.15.71:0/3558698408 wait complete. 2024-02-20T12:58:44.375 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:44.356+0000 7f0ed3d7e700 1 Processor -- start 2024-02-20T12:58:44.375 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:44.356+0000 7f0ed3d7e700 1 -- start start 2024-02-20T12:58:44.375 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:44.356+0000 7f0ed3d7e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0ecc0602c0 0x7f0ecc0f6c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:58:44.375 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:44.356+0000 7f0ed3d7e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ecc0f5100 con 0x7f0ecc0602c0 2024-02-20T12:58:44.375 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:44.356+0000 7f0ed1b1a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0ecc0602c0 0x7f0ecc0f6c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:58:44.375 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:44.356+0000 7f0ed1b1a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0ecc0602c0 0x7f0ecc0f6c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.71:51676/0 (socket says 172.21.15.71:51676) 2024-02-20T12:58:44.375 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:44.356+0000 7f0ed1b1a700 1 -- 172.21.15.71:0/4042755129 learned_addr learned my addr 172.21.15.71:0/4042755129 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-02-20T12:58:44.375 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:44.356+0000 7f0ed1b1a700 1 -- 172.21.15.71:0/4042755129 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ebc005040 con 0x7f0ecc0602c0 2024-02-20T12:58:44.375 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:44.356+0000 7f0ed1b1a700 1 --2- 172.21.15.71:0/4042755129 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0ecc0602c0 0x7f0ecc0f6c80 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 rx=0x7f0ebc013fd0 tx=0x7f0ebc009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:58:44.375 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:44.356+0000 7f0ec27fc700 1 -- 172.21.15.71:0/4042755129 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0ebc018070 con 0x7f0ecc0602c0 2024-02-20T12:58:44.375 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:44.357+0000 7f0ed3d7e700 1 -- 172.21.15.71:0/4042755129 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ecc0f52a0 con 0x7f0ecc0602c0 2024-02-20T12:58:44.376 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:44.357+0000 7f0ec27fc700 1 -- 172.21.15.71:0/4042755129 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(25 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f0ebc007b10 con 0x7f0ecc0602c0 2024-02-20T12:58:44.376 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:44.357+0000 7f0ec27fc700 1 -- 172.21.15.71:0/4042755129 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0ebc01c3c0 con 0x7f0ecc0602c0 2024-02-20T12:58:44.376 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:44.357+0000 7f0ed3d7e700 1 -- 172.21.15.71:0/4042755129 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ecc0f5730 con 0x7f0ecc0602c0 2024-02-20T12:58:44.376 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:44.357+0000 7f0ec27fc700 1 -- 172.21.15.71:0/4042755129 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 37151+0+0 (secure 0 0 0) 0x7f0ebc007c80 con 0x7f0ecc0602c0 2024-02-20T12:58:44.376 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:44.357+0000 7f0ed3d7e700 1 -- 172.21.15.71:0/4042755129 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ecc0591e0 con 0x7f0ecc0602c0 2024-02-20T12:58:44.376 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:44.358+0000 7f0ec27fc700 1 --2- 172.21.15.71:0/4042755129 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f0eb80310e0 0x7f0eb80335a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:58:44.376 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:44.358+0000 7f0ec27fc700 1 -- 172.21.15.71:0/4042755129 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1065+0+0 (secure 0 0 0) 0x7f0ebc042a50 con 0x7f0ecc0602c0 2024-02-20T12:58:44.376 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:44.358+0000 7f0ed1319700 1 --2- 172.21.15.71:0/4042755129 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f0eb80310e0 0x7f0eb80335a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:58:44.376 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:44.358+0000 7f0ed1319700 1 --2- 172.21.15.71:0/4042755129 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f0eb80310e0 0x7f0eb80335a0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 rx=0x7f0ec80096a0 tx=0x7f0ec8006b40).ready entity=mgr.14156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:58:44.376 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:44.363+0000 7f0ec27fc700 1 -- 172.21.15.71:0/4042755129 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+153237 (secure 0 0 0) 0x7f0ebc025b40 con 0x7f0ecc0602c0 2024-02-20T12:58:44.570 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:44.568+0000 7f0ed3d7e700 1 -- 172.21.15.71:0/4042755129 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f0ecc0fc070 con 0x7f0ecc0602c0 2024-02-20T12:58:44.570 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:44.569+0000 7f0ec27fc700 1 -- 172.21.15.71:0/4042755129 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7f0ebc025b40 con 0x7f0ecc0602c0 2024-02-20T12:58:44.570 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-20T12:58:44.571 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":1,"fsid":"680ee126-cfee-11ee-95bb-87774f69a715","modified":"2024-02-20T12:49:44.699059Z","created":"2024-02-20T12:49:44.699059Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi064","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:3300","nonce":0},{"type":"v1","addr":"172.21.15.64:6789","nonce":0}]},"addr":"172.21.15.64:6789/0","public_addr":"172.21.15.64:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-20T12:58:44.572 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:44.570+0000 7f0ed3d7e700 1 -- 172.21.15.71:0/4042755129 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f0eb80310e0 msgr2=0x7f0eb80335a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:58:44.572 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:44.570+0000 7f0ed3d7e700 1 --2- 172.21.15.71:0/4042755129 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f0eb80310e0 0x7f0eb80335a0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 rx=0x7f0ec80096a0 tx=0x7f0ec8006b40).stop 2024-02-20T12:58:44.572 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:44.570+0000 7f0ed3d7e700 1 -- 172.21.15.71:0/4042755129 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0ecc0602c0 msgr2=0x7f0ecc0f6c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:58:44.572 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:44.570+0000 7f0ed3d7e700 1 --2- 172.21.15.71:0/4042755129 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0ecc0602c0 0x7f0ecc0f6c80 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 rx=0x7f0ebc013fd0 tx=0x7f0ebc009b30).stop 2024-02-20T12:58:44.572 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:44.570+0000 7f0ed3d7e700 1 -- 172.21.15.71:0/4042755129 shutdown_connections 2024-02-20T12:58:44.572 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:44.570+0000 7f0ed3d7e700 1 --2- 172.21.15.71:0/4042755129 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f0eb80310e0 0x7f0eb80335a0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:58:44.572 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:44.570+0000 7f0ed3d7e700 1 --2- 172.21.15.71:0/4042755129 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0ecc0602c0 0x7f0ecc0f6c80 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:58:44.573 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:44.570+0000 7f0ed3d7e700 1 -- 172.21.15.71:0/4042755129 >> 172.21.15.71:0/4042755129 conn(0x7f0ecc0f0be0 msgr2=0x7f0ecc0f1630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:58:44.573 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:44.570+0000 7f0ed3d7e700 1 -- 172.21.15.71:0/4042755129 shutdown_connections 2024-02-20T12:58:44.573 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:44.570+0000 7f0ed3d7e700 1 -- 172.21.15.71:0/4042755129 wait complete. 2024-02-20T12:58:44.573 INFO:teuthology.orchestra.run.smithi071.stderr:dumped monmap epoch 1 2024-02-20T12:58:45.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:58:45 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:58:43.779178+0000 mgr.smithi064.vriyor (mgr.14156) 243 : cluster [DBG] pgmap v229: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:58:45.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:58:45 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:58:44.570707+0000 mon.smithi064 (mon.0) 246 : audit [DBG] from='client.? 172.21.15.71:0/4042755129' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-20T12:58:46.290 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-20T12:58:46.291 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/daemon-base:latest-pacific shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 680ee126-cfee-11ee-95bb-87774f69a715 -- ceph mon dump -f json 2024-02-20T12:58:47.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:58:47 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:58:45.779679+0000 mgr.smithi064.vriyor (mgr.14156) 244 : cluster [DBG] pgmap v230: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:58:48.144 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:48.142+0000 7f9bbde27700 1 -- 172.21.15.71:0/456962205 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9bb80f5480 msgr2=0x7f9bb80f58a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:58:48.144 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:48.142+0000 7f9bbde27700 1 --2- 172.21.15.71:0/456962205 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9bb80f5480 0x7f9bb80f58a0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 rx=0x7f9bac009fb0 tx=0x7f9bac009b30).stop 2024-02-20T12:58:48.145 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:48.142+0000 7f9bbde27700 1 -- 172.21.15.71:0/456962205 shutdown_connections 2024-02-20T12:58:48.145 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:48.142+0000 7f9bbde27700 1 --2- 172.21.15.71:0/456962205 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9bb80f5480 0x7f9bb80f58a0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:58:48.145 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:48.142+0000 7f9bbde27700 1 -- 172.21.15.71:0/456962205 >> 172.21.15.71:0/456962205 conn(0x7f9bb80f0be0 msgr2=0x7f9bb80f3000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:58:48.145 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:48.142+0000 7f9bbde27700 1 -- 172.21.15.71:0/456962205 shutdown_connections 2024-02-20T12:58:48.145 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:48.142+0000 7f9bbde27700 1 -- 172.21.15.71:0/456962205 wait complete. 2024-02-20T12:58:48.145 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:48.143+0000 7f9bbde27700 1 Processor -- start 2024-02-20T12:58:48.145 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:48.143+0000 7f9bbde27700 1 -- start start 2024-02-20T12:58:48.145 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:48.143+0000 7f9bbde27700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9bb80f5480 0x7f9bb81067e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:58:48.145 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:48.143+0000 7f9bbde27700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bb8106cd0 con 0x7f9bb80f5480 2024-02-20T12:58:48.145 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:48.143+0000 7f9bb77fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9bb80f5480 0x7f9bb81067e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:58:48.145 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:48.143+0000 7f9bb77fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9bb80f5480 0x7f9bb81067e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.71:40256/0 (socket says 172.21.15.71:40256) 2024-02-20T12:58:48.145 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:48.143+0000 7f9bb77fe700 1 -- 172.21.15.71:0/3974435112 learned_addr learned my addr 172.21.15.71:0/3974435112 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-02-20T12:58:48.145 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:48.144+0000 7f9bb77fe700 1 -- 172.21.15.71:0/3974435112 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bac005040 con 0x7f9bb80f5480 2024-02-20T12:58:48.145 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:48.144+0000 7f9bb77fe700 1 --2- 172.21.15.71:0/3974435112 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9bb80f5480 0x7f9bb81067e0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 rx=0x7f9bac0089f0 tx=0x7f9bac00b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:58:48.146 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:48.144+0000 7f9b9ffff700 1 -- 172.21.15.71:0/3974435112 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9bac009320 con 0x7f9bb80f5480 2024-02-20T12:58:48.146 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:48.144+0000 7f9b9ffff700 1 -- 172.21.15.71:0/3974435112 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(25 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f9bac013ec0 con 0x7f9bb80f5480 2024-02-20T12:58:48.146 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:48.144+0000 7f9bbde27700 1 -- 172.21.15.71:0/3974435112 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9bb8106ed0 con 0x7f9bb80f5480 2024-02-20T12:58:48.146 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:48.144+0000 7f9b9ffff700 1 -- 172.21.15.71:0/3974435112 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9bac014d30 con 0x7f9bb80f5480 2024-02-20T12:58:48.146 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:48.144+0000 7f9bbde27700 1 -- 172.21.15.71:0/3974435112 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9bb8107360 con 0x7f9bb80f5480 2024-02-20T12:58:48.147 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:48.145+0000 7f9bbde27700 1 -- 172.21.15.71:0/3974435112 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9bb8045bb0 con 0x7f9bb80f5480 2024-02-20T12:58:48.147 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:48.145+0000 7f9b9ffff700 1 -- 172.21.15.71:0/3974435112 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 37151+0+0 (secure 0 0 0) 0x7f9bac0095f0 con 0x7f9bb80f5480 2024-02-20T12:58:48.147 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:48.145+0000 7f9b9ffff700 1 --2- 172.21.15.71:0/3974435112 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f9ba0031090 0x7f9ba0033550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:58:48.147 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:48.145+0000 7f9b9ffff700 1 -- 172.21.15.71:0/3974435112 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1065+0+0 (secure 0 0 0) 0x7f9bac0424a0 con 0x7f9bb80f5480 2024-02-20T12:58:48.147 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:48.145+0000 7f9bb6ffd700 1 --2- 172.21.15.71:0/3974435112 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f9ba0031090 0x7f9ba0033550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:58:48.147 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:48.146+0000 7f9bb6ffd700 1 --2- 172.21.15.71:0/3974435112 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f9ba0031090 0x7f9ba0033550 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 rx=0x7f9ba80096a0 tx=0x7f9ba8006b40).ready entity=mgr.14156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:58:48.150 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:48.150+0000 7f9b9ffff700 1 -- 172.21.15.71:0/3974435112 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+153237 (secure 0 0 0) 0x7f9bac01d7f0 con 0x7f9bb80f5480 2024-02-20T12:58:48.356 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:48.356+0000 7f9bbde27700 1 -- 172.21.15.71:0/3974435112 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f9bb8108140 con 0x7f9bb80f5480 2024-02-20T12:58:48.357 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:48.357+0000 7f9b9ffff700 1 -- 172.21.15.71:0/3974435112 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7f9bac00fe40 con 0x7f9bb80f5480 2024-02-20T12:58:48.357 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-20T12:58:48.358 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":1,"fsid":"680ee126-cfee-11ee-95bb-87774f69a715","modified":"2024-02-20T12:49:44.699059Z","created":"2024-02-20T12:49:44.699059Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi064","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:3300","nonce":0},{"type":"v1","addr":"172.21.15.64:6789","nonce":0}]},"addr":"172.21.15.64:6789/0","public_addr":"172.21.15.64:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-20T12:58:48.359 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:48.358+0000 7f9bbde27700 1 -- 172.21.15.71:0/3974435112 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f9ba0031090 msgr2=0x7f9ba0033550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:58:48.359 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:48.358+0000 7f9bbde27700 1 --2- 172.21.15.71:0/3974435112 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f9ba0031090 0x7f9ba0033550 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 rx=0x7f9ba80096a0 tx=0x7f9ba8006b40).stop 2024-02-20T12:58:48.359 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:48.358+0000 7f9bbde27700 1 -- 172.21.15.71:0/3974435112 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9bb80f5480 msgr2=0x7f9bb81067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:58:48.359 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:48.358+0000 7f9bbde27700 1 --2- 172.21.15.71:0/3974435112 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9bb80f5480 0x7f9bb81067e0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 rx=0x7f9bac0089f0 tx=0x7f9bac00b040).stop 2024-02-20T12:58:48.359 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:48.358+0000 7f9bbde27700 1 -- 172.21.15.71:0/3974435112 shutdown_connections 2024-02-20T12:58:48.359 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:48.358+0000 7f9bbde27700 1 --2- 172.21.15.71:0/3974435112 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f9ba0031090 0x7f9ba0033550 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:58:48.359 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:48.358+0000 7f9bbde27700 1 --2- 172.21.15.71:0/3974435112 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9bb80f5480 0x7f9bb81067e0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:58:48.359 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:48.358+0000 7f9bbde27700 1 -- 172.21.15.71:0/3974435112 >> 172.21.15.71:0/3974435112 conn(0x7f9bb80f0be0 msgr2=0x7f9bb805ff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:58:48.359 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:48.358+0000 7f9bbde27700 1 -- 172.21.15.71:0/3974435112 shutdown_connections 2024-02-20T12:58:48.360 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:48.358+0000 7f9bbde27700 1 -- 172.21.15.71:0/3974435112 wait complete. 2024-02-20T12:58:48.360 INFO:teuthology.orchestra.run.smithi071.stderr:dumped monmap epoch 1 2024-02-20T12:58:49.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:58:49 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:58:47.780109+0000 mgr.smithi064.vriyor (mgr.14156) 245 : cluster [DBG] pgmap v231: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:58:49.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:58:49 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:58:48.357656+0000 mon.smithi064 (mon.0) 247 : audit [DBG] from='client.? 172.21.15.71:0/3974435112' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-20T12:58:50.039 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-20T12:58:50.039 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/daemon-base:latest-pacific shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 680ee126-cfee-11ee-95bb-87774f69a715 -- ceph mon dump -f json 2024-02-20T12:58:51.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:58:51 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:58:49.780460+0000 mgr.smithi064.vriyor (mgr.14156) 246 : cluster [DBG] pgmap v232: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:58:51.947 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:51.944+0000 7f6e1a56e700 1 -- 172.21.15.71:0/3528594880 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6e140f54b0 msgr2=0x7f6e140f58d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:58:51.947 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:51.944+0000 7f6e1a56e700 1 --2- 172.21.15.71:0/3528594880 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6e140f54b0 0x7f6e140f58d0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 rx=0x7f6e04009fb0 tx=0x7f6e04009b30).stop 2024-02-20T12:58:51.947 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:51.944+0000 7f6e1a56e700 1 -- 172.21.15.71:0/3528594880 shutdown_connections 2024-02-20T12:58:51.947 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:51.944+0000 7f6e1a56e700 1 --2- 172.21.15.71:0/3528594880 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6e140f54b0 0x7f6e140f58d0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:58:51.947 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:51.944+0000 7f6e1a56e700 1 -- 172.21.15.71:0/3528594880 >> 172.21.15.71:0/3528594880 conn(0x7f6e140f0c10 msgr2=0x7f6e140f3030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:58:51.947 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:51.944+0000 7f6e1a56e700 1 -- 172.21.15.71:0/3528594880 shutdown_connections 2024-02-20T12:58:51.947 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:51.944+0000 7f6e1a56e700 1 -- 172.21.15.71:0/3528594880 wait complete. 2024-02-20T12:58:51.947 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:51.945+0000 7f6e1a56e700 1 Processor -- start 2024-02-20T12:58:51.947 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:51.945+0000 7f6e1a56e700 1 -- start start 2024-02-20T12:58:51.947 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:51.945+0000 7f6e1a56e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6e140f54b0 0x7f6e1410b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:58:51.948 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:51.945+0000 7f6e1a56e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e141072b0 con 0x7f6e140f54b0 2024-02-20T12:58:51.948 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:51.946+0000 7f6e13fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6e140f54b0 0x7f6e1410b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:58:51.948 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:51.946+0000 7f6e13fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6e140f54b0 0x7f6e1410b8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.71:40276/0 (socket says 172.21.15.71:40276) 2024-02-20T12:58:51.948 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:51.946+0000 7f6e13fff700 1 -- 172.21.15.71:0/371119969 learned_addr learned my addr 172.21.15.71:0/371119969 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-02-20T12:58:51.948 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:51.946+0000 7f6e13fff700 1 -- 172.21.15.71:0/371119969 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e04005040 con 0x7f6e140f54b0 2024-02-20T12:58:51.948 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:51.946+0000 7f6e13fff700 1 --2- 172.21.15.71:0/371119969 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6e140f54b0 0x7f6e1410b8c0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 rx=0x7f6e040089f0 tx=0x7f6e0400b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:58:51.948 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:51.946+0000 7f6e10ff9700 1 -- 172.21.15.71:0/371119969 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6e04009320 con 0x7f6e140f54b0 2024-02-20T12:58:51.948 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:51.946+0000 7f6e1a56e700 1 -- 172.21.15.71:0/371119969 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e14060300 con 0x7f6e140f54b0 2024-02-20T12:58:51.948 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:51.946+0000 7f6e10ff9700 1 -- 172.21.15.71:0/371119969 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(25 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f6e04013ec0 con 0x7f6e140f54b0 2024-02-20T12:58:51.948 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:51.946+0000 7f6e1a56e700 1 -- 172.21.15.71:0/371119969 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e1410bdb0 con 0x7f6e140f54b0 2024-02-20T12:58:51.949 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:51.946+0000 7f6e10ff9700 1 -- 172.21.15.71:0/371119969 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6e04014d30 con 0x7f6e140f54b0 2024-02-20T12:58:51.949 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:51.947+0000 7f6e10ff9700 1 -- 172.21.15.71:0/371119969 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 37151+0+0 (secure 0 0 0) 0x7f6e040095f0 con 0x7f6e140f54b0 2024-02-20T12:58:51.949 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:51.947+0000 7f6e10ff9700 1 --2- 172.21.15.71:0/371119969 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f6dfc031090 0x7f6dfc033550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:58:51.949 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:51.947+0000 7f6e10ff9700 1 -- 172.21.15.71:0/371119969 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1065+0+0 (secure 0 0 0) 0x7f6e04042440 con 0x7f6e140f54b0 2024-02-20T12:58:51.949 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:51.947+0000 7f6e1a56e700 1 -- 172.21.15.71:0/371119969 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e14045bb0 con 0x7f6e140f54b0 2024-02-20T12:58:51.949 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:51.947+0000 7f6e137fe700 1 --2- 172.21.15.71:0/371119969 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f6dfc031090 0x7f6dfc033550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:58:51.949 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:51.948+0000 7f6e137fe700 1 --2- 172.21.15.71:0/371119969 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f6dfc031090 0x7f6dfc033550 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 rx=0x7f6e080096a0 tx=0x7f6e08006b40).ready entity=mgr.14156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:58:51.952 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:51.952+0000 7f6e10ff9700 1 -- 172.21.15.71:0/371119969 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+153237 (secure 0 0 0) 0x7f6e0401dbe0 con 0x7f6e140f54b0 2024-02-20T12:58:52.158 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:52.158+0000 7f6e1a56e700 1 -- 172.21.15.71:0/371119969 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f6e140fafa0 con 0x7f6e140f54b0 2024-02-20T12:58:52.159 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:52.159+0000 7f6e10ff9700 1 -- 172.21.15.71:0/371119969 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7f6e140fafa0 con 0x7f6e140f54b0 2024-02-20T12:58:52.160 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-20T12:58:52.160 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":1,"fsid":"680ee126-cfee-11ee-95bb-87774f69a715","modified":"2024-02-20T12:49:44.699059Z","created":"2024-02-20T12:49:44.699059Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi064","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:3300","nonce":0},{"type":"v1","addr":"172.21.15.64:6789","nonce":0}]},"addr":"172.21.15.64:6789/0","public_addr":"172.21.15.64:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-20T12:58:52.161 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:52.161+0000 7f6e1a56e700 1 -- 172.21.15.71:0/371119969 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f6dfc031090 msgr2=0x7f6dfc033550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:58:52.161 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:52.161+0000 7f6e1a56e700 1 --2- 172.21.15.71:0/371119969 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f6dfc031090 0x7f6dfc033550 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 rx=0x7f6e080096a0 tx=0x7f6e08006b40).stop 2024-02-20T12:58:52.161 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:52.161+0000 7f6e1a56e700 1 -- 172.21.15.71:0/371119969 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6e140f54b0 msgr2=0x7f6e1410b8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:58:52.161 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:52.161+0000 7f6e1a56e700 1 --2- 172.21.15.71:0/371119969 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6e140f54b0 0x7f6e1410b8c0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 rx=0x7f6e040089f0 tx=0x7f6e0400b040).stop 2024-02-20T12:58:52.162 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:52.161+0000 7f6e1a56e700 1 -- 172.21.15.71:0/371119969 shutdown_connections 2024-02-20T12:58:52.162 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:52.161+0000 7f6e1a56e700 1 --2- 172.21.15.71:0/371119969 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f6dfc031090 0x7f6dfc033550 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:58:52.162 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:52.161+0000 7f6e1a56e700 1 --2- 172.21.15.71:0/371119969 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6e140f54b0 0x7f6e1410b8c0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:58:52.162 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:52.161+0000 7f6e1a56e700 1 -- 172.21.15.71:0/371119969 >> 172.21.15.71:0/371119969 conn(0x7f6e140f0c10 msgr2=0x7f6e140f1670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:58:52.162 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:52.161+0000 7f6e1a56e700 1 -- 172.21.15.71:0/371119969 shutdown_connections 2024-02-20T12:58:52.162 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:52.161+0000 7f6e1a56e700 1 -- 172.21.15.71:0/371119969 wait complete. 2024-02-20T12:58:52.162 INFO:teuthology.orchestra.run.smithi071.stderr:dumped monmap epoch 1 2024-02-20T12:58:53.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:58:53 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:58:51.780868+0000 mgr.smithi064.vriyor (mgr.14156) 247 : cluster [DBG] pgmap v233: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:58:53.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:58:53 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:58:52.159646+0000 mon.smithi064 (mon.0) 248 : audit [DBG] from='client.? 172.21.15.71:0/371119969' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-20T12:58:53.836 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-20T12:58:53.837 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/daemon-base:latest-pacific shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 680ee126-cfee-11ee-95bb-87774f69a715 -- ceph mon dump -f json 2024-02-20T12:58:54.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:58:54 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:58:53.850575+0000 mon.smithi064 (mon.0) 249 : audit [INF] from='mgr.14156 172.21.15.64:0/3284974785' entity='mgr.smithi064.vriyor' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi064.vriyor/trash_purge_schedule"}]: dispatch 2024-02-20T12:58:54.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:58:54 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:58:53.850838+0000 mon.smithi064 (mon.0) 250 : audit [INF] from='mgr.14156 172.21.15.64:0/3284974785' entity='mgr.smithi064.vriyor' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi064.vriyor/mirror_snapshot_schedule"}]: dispatch 2024-02-20T12:58:55.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:58:55 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:58:53.781220+0000 mgr.smithi064.vriyor (mgr.14156) 248 : cluster [DBG] pgmap v234: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:58:55.952 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:55.951+0000 7f84c2702700 1 -- 172.21.15.71:0/2987858470 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f84bc0f5480 msgr2=0x7f84bc0f58a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:58:55.953 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:55.951+0000 7f84c2702700 1 --2- 172.21.15.71:0/2987858470 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f84bc0f5480 0x7f84bc0f58a0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 rx=0x7f84a4009fb0 tx=0x7f84a4009b30).stop 2024-02-20T12:58:55.953 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:55.951+0000 7f84c2702700 1 -- 172.21.15.71:0/2987858470 shutdown_connections 2024-02-20T12:58:55.953 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:55.951+0000 7f84c2702700 1 --2- 172.21.15.71:0/2987858470 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f84bc0f5480 0x7f84bc0f58a0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:58:55.953 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:55.951+0000 7f84c2702700 1 -- 172.21.15.71:0/2987858470 >> 172.21.15.71:0/2987858470 conn(0x7f84bc0f0be0 msgr2=0x7f84bc0f3000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:58:55.953 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:55.951+0000 7f84c2702700 1 -- 172.21.15.71:0/2987858470 shutdown_connections 2024-02-20T12:58:55.953 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:55.951+0000 7f84c2702700 1 -- 172.21.15.71:0/2987858470 wait complete. 2024-02-20T12:58:55.953 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:55.951+0000 7f84c2702700 1 Processor -- start 2024-02-20T12:58:55.954 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:55.952+0000 7f84c2702700 1 -- start start 2024-02-20T12:58:55.954 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:55.952+0000 7f84c2702700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f84bc0f5480 0x7f84bc1056f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:58:55.954 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:55.952+0000 7f84c2702700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84bc105be0 con 0x7f84bc0f5480 2024-02-20T12:58:55.954 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:55.952+0000 7f84bbfff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f84bc0f5480 0x7f84bc1056f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:58:55.954 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:55.952+0000 7f84bbfff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f84bc0f5480 0x7f84bc1056f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.71:40294/0 (socket says 172.21.15.71:40294) 2024-02-20T12:58:55.954 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:55.952+0000 7f84bbfff700 1 -- 172.21.15.71:0/4026720254 learned_addr learned my addr 172.21.15.71:0/4026720254 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-02-20T12:58:55.954 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:55.952+0000 7f84bbfff700 1 -- 172.21.15.71:0/4026720254 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84a4005040 con 0x7f84bc0f5480 2024-02-20T12:58:55.954 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:55.952+0000 7f84bbfff700 1 --2- 172.21.15.71:0/4026720254 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f84bc0f5480 0x7f84bc1056f0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 rx=0x7f84a40089f0 tx=0x7f84a400b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:58:55.956 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:55.953+0000 7f84b8ff9700 1 -- 172.21.15.71:0/4026720254 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f84a4009320 con 0x7f84bc0f5480 2024-02-20T12:58:55.956 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:55.953+0000 7f84c2702700 1 -- 172.21.15.71:0/4026720254 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84bc105de0 con 0x7f84bc0f5480 2024-02-20T12:58:55.956 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:55.953+0000 7f84b8ff9700 1 -- 172.21.15.71:0/4026720254 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(25 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f84a4013ec0 con 0x7f84bc0f5480 2024-02-20T12:58:55.956 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:55.953+0000 7f84b8ff9700 1 -- 172.21.15.71:0/4026720254 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f84a4014d30 con 0x7f84bc0f5480 2024-02-20T12:58:55.956 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:55.953+0000 7f84c2702700 1 -- 172.21.15.71:0/4026720254 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84bc106270 con 0x7f84bc0f5480 2024-02-20T12:58:55.956 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:55.953+0000 7f84b8ff9700 1 -- 172.21.15.71:0/4026720254 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 37151+0+0 (secure 0 0 0) 0x7f84a40263a0 con 0x7f84bc0f5480 2024-02-20T12:58:55.956 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:55.954+0000 7f84b8ff9700 1 --2- 172.21.15.71:0/4026720254 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f84a8031090 0x7f84a8033550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:58:55.956 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:55.954+0000 7f84b8ff9700 1 -- 172.21.15.71:0/4026720254 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1065+0+0 (secure 0 0 0) 0x7f84a40423c0 con 0x7f84bc0f5480 2024-02-20T12:58:55.956 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:55.954+0000 7f84c2702700 1 -- 172.21.15.71:0/4026720254 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84bc045bb0 con 0x7f84bc0f5480 2024-02-20T12:58:55.956 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:55.954+0000 7f84bb7fe700 1 --2- 172.21.15.71:0/4026720254 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f84a8031090 0x7f84a8033550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:58:55.956 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:55.954+0000 7f84bb7fe700 1 --2- 172.21.15.71:0/4026720254 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f84a8031090 0x7f84a8033550 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 rx=0x7f84ac0096a0 tx=0x7f84ac006b40).ready entity=mgr.14156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:58:55.959 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:55.958+0000 7f84b8ff9700 1 -- 172.21.15.71:0/4026720254 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+153237 (secure 0 0 0) 0x7f84a40266d0 con 0x7f84bc0f5480 2024-02-20T12:58:56.167 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:56.166+0000 7f84c2702700 1 -- 172.21.15.71:0/4026720254 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f84bc0fb0c0 con 0x7f84bc0f5480 2024-02-20T12:58:56.167 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:56.167+0000 7f84b8ff9700 1 -- 172.21.15.71:0/4026720254 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7f84a4012330 con 0x7f84bc0f5480 2024-02-20T12:58:56.168 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-20T12:58:56.168 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":1,"fsid":"680ee126-cfee-11ee-95bb-87774f69a715","modified":"2024-02-20T12:49:44.699059Z","created":"2024-02-20T12:49:44.699059Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi064","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:3300","nonce":0},{"type":"v1","addr":"172.21.15.64:6789","nonce":0}]},"addr":"172.21.15.64:6789/0","public_addr":"172.21.15.64:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-20T12:58:56.169 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:56.168+0000 7f84c2702700 1 -- 172.21.15.71:0/4026720254 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f84a8031090 msgr2=0x7f84a8033550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:58:56.169 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:56.168+0000 7f84c2702700 1 --2- 172.21.15.71:0/4026720254 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f84a8031090 0x7f84a8033550 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 rx=0x7f84ac0096a0 tx=0x7f84ac006b40).stop 2024-02-20T12:58:56.169 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:56.168+0000 7f84c2702700 1 -- 172.21.15.71:0/4026720254 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f84bc0f5480 msgr2=0x7f84bc1056f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:58:56.169 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:56.168+0000 7f84c2702700 1 --2- 172.21.15.71:0/4026720254 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f84bc0f5480 0x7f84bc1056f0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 rx=0x7f84a40089f0 tx=0x7f84a400b040).stop 2024-02-20T12:58:56.169 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:56.169+0000 7f84c2702700 1 -- 172.21.15.71:0/4026720254 shutdown_connections 2024-02-20T12:58:56.169 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:56.169+0000 7f84c2702700 1 --2- 172.21.15.71:0/4026720254 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f84a8031090 0x7f84a8033550 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:58:56.169 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:56.169+0000 7f84c2702700 1 --2- 172.21.15.71:0/4026720254 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f84bc0f5480 0x7f84bc1056f0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:58:56.169 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:56.169+0000 7f84c2702700 1 -- 172.21.15.71:0/4026720254 >> 172.21.15.71:0/4026720254 conn(0x7f84bc0f0be0 msgr2=0x7f84bc0f16d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:58:56.169 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:56.169+0000 7f84c2702700 1 -- 172.21.15.71:0/4026720254 shutdown_connections 2024-02-20T12:58:56.170 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:58:56.169+0000 7f84c2702700 1 -- 172.21.15.71:0/4026720254 wait complete. 2024-02-20T12:58:56.170 INFO:teuthology.orchestra.run.smithi071.stderr:dumped monmap epoch 1 2024-02-20T12:58:57.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:58:57 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:58:55.781503+0000 mgr.smithi064.vriyor (mgr.14156) 249 : cluster [DBG] pgmap v235: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:58:57.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:58:57 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:58:56.167806+0000 mon.smithi064 (mon.0) 251 : audit [DBG] from='client.? 172.21.15.71:0/4026720254' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-20T12:58:58.038 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-20T12:58:58.038 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/daemon-base:latest-pacific shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 680ee126-cfee-11ee-95bb-87774f69a715 -- ceph mon dump -f json 2024-02-20T12:58:59.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:58:59 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:58:57.781822+0000 mgr.smithi064.vriyor (mgr.14156) 250 : cluster [DBG] pgmap v236: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:59:00.086 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:00.084+0000 7f48cac79700 1 -- 172.21.15.71:0/1312930886 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f48c40f52d0 msgr2=0x7f48c40f56f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:59:00.087 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:00.084+0000 7f48cac79700 1 --2- 172.21.15.71:0/1312930886 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f48c40f52d0 0x7f48c40f56f0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 rx=0x7f48c0009fb0 tx=0x7f48c0009b30).stop 2024-02-20T12:59:00.087 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:00.084+0000 7f48cac79700 1 -- 172.21.15.71:0/1312930886 shutdown_connections 2024-02-20T12:59:00.087 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:00.084+0000 7f48cac79700 1 --2- 172.21.15.71:0/1312930886 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f48c40f52d0 0x7f48c40f56f0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:59:00.087 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:00.084+0000 7f48cac79700 1 -- 172.21.15.71:0/1312930886 >> 172.21.15.71:0/1312930886 conn(0x7f48c40f0a30 msgr2=0x7f48c40f2e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:59:00.087 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:00.085+0000 7f48cac79700 1 -- 172.21.15.71:0/1312930886 shutdown_connections 2024-02-20T12:59:00.087 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:00.085+0000 7f48cac79700 1 -- 172.21.15.71:0/1312930886 wait complete. 2024-02-20T12:59:00.087 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:00.085+0000 7f48cac79700 1 Processor -- start 2024-02-20T12:59:00.087 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:00.085+0000 7f48cac79700 1 -- start start 2024-02-20T12:59:00.087 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:00.085+0000 7f48cac79700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f48c40f52d0 0x7f48c4109310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:59:00.087 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:00.085+0000 7f48cac79700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48c4060320 con 0x7f48c40f52d0 2024-02-20T12:59:00.087 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:00.085+0000 7f48c9c77700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f48c40f52d0 0x7f48c4109310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:59:00.087 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:00.085+0000 7f48c9c77700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f48c40f52d0 0x7f48c4109310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.71:52402/0 (socket says 172.21.15.71:52402) 2024-02-20T12:59:00.087 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:00.085+0000 7f48c9c77700 1 -- 172.21.15.71:0/401347666 learned_addr learned my addr 172.21.15.71:0/401347666 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-02-20T12:59:00.088 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:00.086+0000 7f48c9c77700 1 -- 172.21.15.71:0/401347666 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48c0005040 con 0x7f48c40f52d0 2024-02-20T12:59:00.088 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:00.086+0000 7f48c9c77700 1 --2- 172.21.15.71:0/401347666 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f48c40f52d0 0x7f48c4109310 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 rx=0x7f48c00089f0 tx=0x7f48c000b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:59:00.088 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:00.086+0000 7f48ba7fc700 1 -- 172.21.15.71:0/401347666 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f48c0009320 con 0x7f48c40f52d0 2024-02-20T12:59:00.088 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:00.086+0000 7f48cac79700 1 -- 172.21.15.71:0/401347666 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48c4060520 con 0x7f48c40f52d0 2024-02-20T12:59:00.088 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:00.086+0000 7f48ba7fc700 1 -- 172.21.15.71:0/401347666 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(25 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f48c0013ec0 con 0x7f48c40f52d0 2024-02-20T12:59:00.088 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:00.086+0000 7f48ba7fc700 1 -- 172.21.15.71:0/401347666 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f48c0014d30 con 0x7f48c40f52d0 2024-02-20T12:59:00.088 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:00.086+0000 7f48cac79700 1 -- 172.21.15.71:0/401347666 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48c41099c0 con 0x7f48c40f52d0 2024-02-20T12:59:00.089 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:00.087+0000 7f48ba7fc700 1 -- 172.21.15.71:0/401347666 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 37151+0+0 (secure 0 0 0) 0x7f48c00095f0 con 0x7f48c40f52d0 2024-02-20T12:59:00.089 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:00.087+0000 7f48cac79700 1 -- 172.21.15.71:0/401347666 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48c4045bb0 con 0x7f48c40f52d0 2024-02-20T12:59:00.089 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:00.087+0000 7f48ba7fc700 1 --2- 172.21.15.71:0/401347666 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f48b00310e0 0x7f48b00335a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:59:00.089 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:00.087+0000 7f48ba7fc700 1 -- 172.21.15.71:0/401347666 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1065+0+0 (secure 0 0 0) 0x7f48c00424a0 con 0x7f48c40f52d0 2024-02-20T12:59:00.089 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:00.087+0000 7f48c9476700 1 --2- 172.21.15.71:0/401347666 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f48b00310e0 0x7f48b00335a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:59:00.089 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:00.088+0000 7f48c9476700 1 --2- 172.21.15.71:0/401347666 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f48b00310e0 0x7f48b00335a0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 rx=0x7f48b40096a0 tx=0x7f48b4006b40).ready entity=mgr.14156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:59:00.092 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:00.092+0000 7f48ba7fc700 1 -- 172.21.15.71:0/401347666 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+153237 (secure 0 0 0) 0x7f48c001d980 con 0x7f48c40f52d0 2024-02-20T12:59:00.301 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:00.300+0000 7f48cac79700 1 -- 172.21.15.71:0/401347666 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f48c410a1e0 con 0x7f48c40f52d0 2024-02-20T12:59:00.301 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:00.301+0000 7f48ba7fc700 1 -- 172.21.15.71:0/401347666 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7f48c001d380 con 0x7f48c40f52d0 2024-02-20T12:59:00.301 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-20T12:59:00.301 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":1,"fsid":"680ee126-cfee-11ee-95bb-87774f69a715","modified":"2024-02-20T12:49:44.699059Z","created":"2024-02-20T12:49:44.699059Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi064","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:3300","nonce":0},{"type":"v1","addr":"172.21.15.64:6789","nonce":0}]},"addr":"172.21.15.64:6789/0","public_addr":"172.21.15.64:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-20T12:59:00.303 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:00.302+0000 7f48cac79700 1 -- 172.21.15.71:0/401347666 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f48b00310e0 msgr2=0x7f48b00335a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:59:00.303 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:00.302+0000 7f48cac79700 1 --2- 172.21.15.71:0/401347666 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f48b00310e0 0x7f48b00335a0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 rx=0x7f48b40096a0 tx=0x7f48b4006b40).stop 2024-02-20T12:59:00.303 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:00.302+0000 7f48cac79700 1 -- 172.21.15.71:0/401347666 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f48c40f52d0 msgr2=0x7f48c4109310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:59:00.303 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:00.302+0000 7f48cac79700 1 --2- 172.21.15.71:0/401347666 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f48c40f52d0 0x7f48c4109310 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 rx=0x7f48c00089f0 tx=0x7f48c000b040).stop 2024-02-20T12:59:00.303 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:00.302+0000 7f48cac79700 1 -- 172.21.15.71:0/401347666 shutdown_connections 2024-02-20T12:59:00.303 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:00.302+0000 7f48cac79700 1 --2- 172.21.15.71:0/401347666 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f48b00310e0 0x7f48b00335a0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:59:00.303 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:00.302+0000 7f48cac79700 1 --2- 172.21.15.71:0/401347666 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f48c40f52d0 0x7f48c4109310 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:59:00.303 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:00.302+0000 7f48cac79700 1 -- 172.21.15.71:0/401347666 >> 172.21.15.71:0/401347666 conn(0x7f48c40f0a30 msgr2=0x7f48c40f1550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:59:00.303 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:00.302+0000 7f48cac79700 1 -- 172.21.15.71:0/401347666 shutdown_connections 2024-02-20T12:59:00.303 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:00.302+0000 7f48cac79700 1 -- 172.21.15.71:0/401347666 wait complete. 2024-02-20T12:59:00.304 INFO:teuthology.orchestra.run.smithi071.stderr:dumped monmap epoch 1 2024-02-20T12:59:01.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:59:01 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:58:59.782196+0000 mgr.smithi064.vriyor (mgr.14156) 251 : cluster [DBG] pgmap v237: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:59:01.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:59:01 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:59:00.301407+0000 mon.smithi064 (mon.0) 252 : audit [DBG] from='client.? 172.21.15.71:0/401347666' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-20T12:59:01.996 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-20T12:59:01.996 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/daemon-base:latest-pacific shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 680ee126-cfee-11ee-95bb-87774f69a715 -- ceph mon dump -f json 2024-02-20T12:59:03.563 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:59:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:59:01.782503+0000 mgr.smithi064.vriyor (mgr.14156) 252 : cluster [DBG] pgmap v238: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:59:03.563 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:59:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cephadm 2024-02-20T12:59:02.105728+0000 mgr.smithi064.vriyor (mgr.14156) 253 : cephadm [ERR] cephadm exited with an error code: 1, stderr:Non-zero exit code 125 from /bin/podman container inspect --format {{.State.Status}} ceph-680ee126-cfee-11ee-95bb-87774f69a715-node-exporter.smithi064 2024-02-20T12:59:03.563 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:59:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: /bin/podman: stderr Error: no such container ceph-680ee126-cfee-11ee-95bb-87774f69a715-node-exporter.smithi064 2024-02-20T12:59:03.563 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:59:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: Deploy daemon node-exporter.smithi064 ... 2024-02-20T12:59:03.563 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:59:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: Verifying port 9100 ... 2024-02-20T12:59:03.563 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:59:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: Non-zero exit code 1 from systemctl start ceph-680ee126-cfee-11ee-95bb-87774f69a715@node-exporter.smithi064 2024-02-20T12:59:03.564 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:59:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: systemctl: stderr Job for ceph-680ee126-cfee-11ee-95bb-87774f69a715@node-exporter.smithi064.service failed because a timeout was exceeded. 2024-02-20T12:59:03.564 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:59:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: systemctl: stderr See "systemctl status ceph-680ee126-cfee-11ee-95bb-87774f69a715@node-exporter.smithi064.service" and "journalctl -xe" for details. 2024-02-20T12:59:03.564 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:59:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: Traceback (most recent call last): 2024-02-20T12:59:03.564 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:59:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: File "/var/lib/ceph/680ee126-cfee-11ee-95bb-87774f69a715/cephadm.d4237e4639c108308fe13147b1c08af93c3d5724d9ff21ae797eb4b78fea3931", line 8230, in 2024-02-20T12:59:03.564 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:59:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: main() 2024-02-20T12:59:03.564 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:59:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: File "/var/lib/ceph/680ee126-cfee-11ee-95bb-87774f69a715/cephadm.d4237e4639c108308fe13147b1c08af93c3d5724d9ff21ae797eb4b78fea3931", line 8218, in main 2024-02-20T12:59:03.564 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:59:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: r = ctx.func(ctx) 2024-02-20T12:59:03.564 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:59:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: File "/var/lib/ceph/680ee126-cfee-11ee-95bb-87774f69a715/cephadm.d4237e4639c108308fe13147b1c08af93c3d5724d9ff21ae797eb4b78fea3931", line 1759, in _default_image 2024-02-20T12:59:03.564 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:59:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: return func(ctx) 2024-02-20T12:59:03.564 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:59:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: File "/var/lib/ceph/680ee126-cfee-11ee-95bb-87774f69a715/cephadm.d4237e4639c108308fe13147b1c08af93c3d5724d9ff21ae797eb4b78fea3931", line 4348, in command_deploy 2024-02-20T12:59:03.564 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:59:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: ports=daemon_ports) 2024-02-20T12:59:03.564 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:59:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: File "/var/lib/ceph/680ee126-cfee-11ee-95bb-87774f69a715/cephadm.d4237e4639c108308fe13147b1c08af93c3d5724d9ff21ae797eb4b78fea3931", line 2632, in deploy_daemon 2024-02-20T12:59:03.564 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:59:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: c, osd_fsid=osd_fsid, ports=ports) 2024-02-20T12:59:03.565 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:59:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: File "/var/lib/ceph/680ee126-cfee-11ee-95bb-87774f69a715/cephadm.d4237e4639c108308fe13147b1c08af93c3d5724d9ff21ae797eb4b78fea3931", line 2821, in deploy_daemon_units 2024-02-20T12:59:03.565 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:59:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: call_throws(ctx, ['systemctl', 'start', unit_name]) 2024-02-20T12:59:03.565 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:59:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: File "/var/lib/ceph/680ee126-cfee-11ee-95bb-87774f69a715/cephadm.d4237e4639c108308fe13147b1c08af93c3d5724d9ff21ae797eb4b78fea3931", line 1453, in call_throws 2024-02-20T12:59:03.565 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:59:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: raise RuntimeError('Failed command: %s' % ' '.join(command)) 2024-02-20T12:59:03.565 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:59:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: RuntimeError: Failed command: systemctl start ceph-680ee126-cfee-11ee-95bb-87774f69a715@node-exporter.smithi064 2024-02-20T12:59:03.565 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:59:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: Traceback (most recent call last): 2024-02-20T12:59:03.565 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:59:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: File "/usr/share/ceph/mgr/cephadm/serve.py", line 1347, in _remote_connection 2024-02-20T12:59:03.565 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:59:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: yield (conn, connr) 2024-02-20T12:59:03.565 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:59:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: File "/usr/share/ceph/mgr/cephadm/serve.py", line 1244, in _run_cephadm 2024-02-20T12:59:03.565 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:59:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: code, '\n'.join(err))) 2024-02-20T12:59:03.565 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:59:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: orchestrator._interface.OrchestratorError: cephadm exited with an error code: 1, stderr:Non-zero exit code 125 from /bin/podman container inspect --format {{.State.Status}} ceph-680ee126-cfee-11ee-95bb-87774f69a715-node-exporter.smithi064 2024-02-20T12:59:03.565 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:59:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: /bin/podman: stderr Error: no such container ceph-680ee126-cfee-11ee-95bb-87774f69a715-node-exporter.smithi064 2024-02-20T12:59:03.565 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:59:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: Deploy daemon node-exporter.smithi064 ... 2024-02-20T12:59:03.565 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:59:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: Verifying port 9100 ... 2024-02-20T12:59:03.565 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:59:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: Non-zero exit code 1 from systemctl start ceph-680ee126-cfee-11ee-95bb-87774f69a715@node-exporter.smithi064 2024-02-20T12:59:03.566 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:59:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: systemctl: stderr Job for ceph-680ee126-cfee-11ee-95bb-87774f69a715@node-exporter.smithi064.service failed because a timeout was exceeded. 2024-02-20T12:59:03.566 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:59:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: systemctl: stderr See "systemctl status ceph-680ee126-cfee-11ee-95bb-87774f69a715@node-exporter.smithi064.service" and "journalctl -xe" for details. 2024-02-20T12:59:03.566 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:59:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: Traceback (most recent call last): 2024-02-20T12:59:03.566 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:59:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: File "/var/lib/ceph/680ee126-cfee-11ee-95bb-87774f69a715/cephadm.d4237e4639c108308fe13147b1c08af93c3d5724d9ff21ae797eb4b78fea3931", line 8230, in 2024-02-20T12:59:03.566 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:59:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: main() 2024-02-20T12:59:03.566 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:59:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: File "/var/lib/ceph/680ee126-cfee-11ee-95bb-87774f69a715/cephadm.d4237e4639c108308fe13147b1c08af93c3d5724d9ff21ae797eb4b78fea3931", line 8218, in main 2024-02-20T12:59:03.566 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:59:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: r = ctx.func(ctx) 2024-02-20T12:59:03.566 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:59:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: File "/var/lib/ceph/680ee126-cfee-11ee-95bb-87774f69a715/cephadm.d4237e4639c108308fe13147b1c08af93c3d5724d9ff21ae797eb4b78fea3931", line 1759, in _default_image 2024-02-20T12:59:03.566 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:59:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: return func(ctx) 2024-02-20T12:59:03.566 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:59:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: File "/var/lib/ceph/680ee126-cfee-11ee-95bb-87774f69a715/cephadm.d4237e4639c108308fe13147b1c08af93c3d5724d9ff21ae797eb4b78fea3931", line 4348, in command_deploy 2024-02-20T12:59:03.566 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:59:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: ports=daemon_ports) 2024-02-20T12:59:03.566 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:59:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: File "/var/lib/ceph/680ee126-cfee-11ee-95bb-87774f69a715/cephadm.d4237e4639c108308fe13147b1c08af93c3d5724d9ff21ae797eb4b78fea3931", line 2632, in deploy_daemon 2024-02-20T12:59:03.566 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:59:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: c, osd_fsid=osd_fsid, ports=ports) 2024-02-20T12:59:03.566 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:59:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: File "/var/lib/ceph/680ee126-cfee-11ee-95bb-87774f69a715/cephadm.d4237e4639c108308fe13147b1c08af93c3d5724d9ff21ae797eb4b78fea3931", line 2821, in deploy_daemon_units 2024-02-20T12:59:03.567 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:59:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: call_throws(ctx, ['systemctl', 'start', unit_name]) 2024-02-20T12:59:03.567 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:59:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: File "/var/lib/ceph/680ee126-cfee-11ee-95bb-87774f69a715/cephadm.d4237e4639c108308fe13147b1c08af93c3d5724d9ff21ae797eb4b78fea3931", line 1453, in call_throws 2024-02-20T12:59:03.567 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:59:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: raise RuntimeError('Failed command: %s' % ' '.join(command)) 2024-02-20T12:59:03.567 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:59:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: RuntimeError: Failed command: systemctl start ceph-680ee126-cfee-11ee-95bb-87774f69a715@node-exporter.smithi064 2024-02-20T12:59:03.567 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:59:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cephadm 2024-02-20T12:59:02.105955+0000 mgr.smithi064.vriyor (mgr.14156) 254 : cephadm [ERR] Failed while placing node-exporter.smithi064 on smithi064: cephadm exited with an error code: 1, stderr:Non-zero exit code 125 from /bin/podman container inspect --format {{.State.Status}} ceph-680ee126-cfee-11ee-95bb-87774f69a715-node-exporter.smithi064 2024-02-20T12:59:03.567 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:59:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: /bin/podman: stderr Error: no such container ceph-680ee126-cfee-11ee-95bb-87774f69a715-node-exporter.smithi064 2024-02-20T12:59:03.567 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:59:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: Deploy daemon node-exporter.smithi064 ... 2024-02-20T12:59:03.567 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:59:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: Verifying port 9100 ... 2024-02-20T12:59:03.567 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:59:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: Non-zero exit code 1 from systemctl start ceph-680ee126-cfee-11ee-95bb-87774f69a715@node-exporter.smithi064 2024-02-20T12:59:03.567 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:59:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: systemctl: stderr Job for ceph-680ee126-cfee-11ee-95bb-87774f69a715@node-exporter.smithi064.service failed because a timeout was exceeded. 2024-02-20T12:59:03.568 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:59:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: systemctl: stderr See "systemctl status ceph-680ee126-cfee-11ee-95bb-87774f69a715@node-exporter.smithi064.service" and "journalctl -xe" for details. 2024-02-20T12:59:03.568 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:59:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: Traceback (most recent call last): 2024-02-20T12:59:03.568 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:59:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: File "/var/lib/ceph/680ee126-cfee-11ee-95bb-87774f69a715/cephadm.d4237e4639c108308fe13147b1c08af93c3d5724d9ff21ae797eb4b78fea3931", line 8230, in 2024-02-20T12:59:03.568 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:59:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: main() 2024-02-20T12:59:03.568 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:59:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: File "/var/lib/ceph/680ee126-cfee-11ee-95bb-87774f69a715/cephadm.d4237e4639c108308fe13147b1c08af93c3d5724d9ff21ae797eb4b78fea3931", line 8218, in main 2024-02-20T12:59:03.568 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:59:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: r = ctx.func(ctx) 2024-02-20T12:59:03.568 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:59:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: File "/var/lib/ceph/680ee126-cfee-11ee-95bb-87774f69a715/cephadm.d4237e4639c108308fe13147b1c08af93c3d5724d9ff21ae797eb4b78fea3931", line 1759, in _default_image 2024-02-20T12:59:03.568 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:59:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: return func(ctx) 2024-02-20T12:59:03.568 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:59:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: File "/var/lib/ceph/680ee126-cfee-11ee-95bb-87774f69a715/cephadm.d4237e4639c108308fe13147b1c08af93c3d5724d9ff21ae797eb4b78fea3931", line 4348, in command_deploy 2024-02-20T12:59:03.568 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:59:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: ports=daemon_ports) 2024-02-20T12:59:03.568 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:59:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: File "/var/lib/ceph/680ee126-cfee-11ee-95bb-87774f69a715/cephadm.d4237e4639c108308fe13147b1c08af93c3d5724d9ff21ae797eb4b78fea3931", line 2632, in deploy_daemon 2024-02-20T12:59:03.569 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:59:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: c, osd_fsid=osd_fsid, ports=ports) 2024-02-20T12:59:03.569 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:59:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: File "/var/lib/ceph/680ee126-cfee-11ee-95bb-87774f69a715/cephadm.d4237e4639c108308fe13147b1c08af93c3d5724d9ff21ae797eb4b78fea3931", line 2821, in deploy_daemon_units 2024-02-20T12:59:03.569 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:59:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: call_throws(ctx, ['systemctl', 'start', unit_name]) 2024-02-20T12:59:03.569 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:59:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: File "/var/lib/ceph/680ee126-cfee-11ee-95bb-87774f69a715/cephadm.d4237e4639c108308fe13147b1c08af93c3d5724d9ff21ae797eb4b78fea3931", line 1453, in call_throws 2024-02-20T12:59:03.569 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:59:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: raise RuntimeError('Failed command: %s' % ' '.join(command)) 2024-02-20T12:59:03.569 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:59:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: RuntimeError: Failed command: systemctl start ceph-680ee126-cfee-11ee-95bb-87774f69a715@node-exporter.smithi064 2024-02-20T12:59:03.569 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:59:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cephadm 2024-02-20T12:59:02.112157+0000 mgr.smithi064.vriyor (mgr.14156) 255 : cephadm [INF] Deploying daemon prometheus.smithi064 on smithi064 2024-02-20T12:59:03.945 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:03.943+0000 7f5ddecd1700 1 -- 172.21.15.71:0/2851137301 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5dd80f52d0 msgr2=0x7f5dd80f56f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:59:03.946 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:03.943+0000 7f5ddecd1700 1 --2- 172.21.15.71:0/2851137301 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5dd80f52d0 0x7f5dd80f56f0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 rx=0x7f5dcc009fb0 tx=0x7f5dcc009b30).stop 2024-02-20T12:59:03.946 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:03.943+0000 7f5ddecd1700 1 -- 172.21.15.71:0/2851137301 shutdown_connections 2024-02-20T12:59:03.946 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:03.943+0000 7f5ddecd1700 1 --2- 172.21.15.71:0/2851137301 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5dd80f52d0 0x7f5dd80f56f0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:59:03.946 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:03.943+0000 7f5ddecd1700 1 -- 172.21.15.71:0/2851137301 >> 172.21.15.71:0/2851137301 conn(0x7f5dd80f0a30 msgr2=0x7f5dd80f2e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:59:03.946 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:03.943+0000 7f5ddecd1700 1 -- 172.21.15.71:0/2851137301 shutdown_connections 2024-02-20T12:59:03.946 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:03.943+0000 7f5ddecd1700 1 -- 172.21.15.71:0/2851137301 wait complete. 2024-02-20T12:59:03.946 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:03.944+0000 7f5ddecd1700 1 Processor -- start 2024-02-20T12:59:03.946 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:03.944+0000 7f5ddecd1700 1 -- start start 2024-02-20T12:59:03.946 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:03.944+0000 7f5ddecd1700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5dd80f52d0 0x7f5dd8105570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:59:03.947 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:03.944+0000 7f5ddecd1700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dd8105a60 con 0x7f5dd80f52d0 2024-02-20T12:59:03.947 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:03.944+0000 7f5dddccf700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5dd80f52d0 0x7f5dd8105570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:59:03.947 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:03.944+0000 7f5dddccf700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5dd80f52d0 0x7f5dd8105570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.71:52412/0 (socket says 172.21.15.71:52412) 2024-02-20T12:59:03.947 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:03.944+0000 7f5dddccf700 1 -- 172.21.15.71:0/882079982 learned_addr learned my addr 172.21.15.71:0/882079982 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-02-20T12:59:03.947 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:03.944+0000 7f5dddccf700 1 -- 172.21.15.71:0/882079982 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5dcc005040 con 0x7f5dd80f52d0 2024-02-20T12:59:03.947 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:03.944+0000 7f5dddccf700 1 --2- 172.21.15.71:0/882079982 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5dd80f52d0 0x7f5dd8105570 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 rx=0x7f5dcc0089f0 tx=0x7f5dcc00b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:59:03.947 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:03.945+0000 7f5dca7fc700 1 -- 172.21.15.71:0/882079982 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5dcc009320 con 0x7f5dd80f52d0 2024-02-20T12:59:03.947 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:03.945+0000 7f5ddecd1700 1 -- 172.21.15.71:0/882079982 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5dd8105c60 con 0x7f5dd80f52d0 2024-02-20T12:59:03.947 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:03.945+0000 7f5dca7fc700 1 -- 172.21.15.71:0/882079982 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(25 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f5dcc013ec0 con 0x7f5dd80f52d0 2024-02-20T12:59:03.947 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:03.945+0000 7f5dca7fc700 1 -- 172.21.15.71:0/882079982 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5dcc014d30 con 0x7f5dd80f52d0 2024-02-20T12:59:03.947 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:03.945+0000 7f5ddecd1700 1 -- 172.21.15.71:0/882079982 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5dd81060f0 con 0x7f5dd80f52d0 2024-02-20T12:59:03.948 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:03.945+0000 7f5dca7fc700 1 -- 172.21.15.71:0/882079982 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 37151+0+0 (secure 0 0 0) 0x7f5dcc0263a0 con 0x7f5dd80f52d0 2024-02-20T12:59:03.948 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:03.946+0000 7f5dca7fc700 1 --2- 172.21.15.71:0/882079982 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f5dc4031090 0x7f5dc4033550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:59:03.949 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:03.946+0000 7f5dca7fc700 1 -- 172.21.15.71:0/882079982 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1065+0+0 (secure 0 0 0) 0x7f5dcc0423d0 con 0x7f5dd80f52d0 2024-02-20T12:59:03.949 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:03.946+0000 7f5ddecd1700 1 -- 172.21.15.71:0/882079982 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5dd8045bb0 con 0x7f5dd80f52d0 2024-02-20T12:59:03.949 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:03.946+0000 7f5ddd4ce700 1 --2- 172.21.15.71:0/882079982 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f5dc4031090 0x7f5dc4033550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:59:03.949 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:03.946+0000 7f5ddd4ce700 1 --2- 172.21.15.71:0/882079982 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f5dc4031090 0x7f5dc4033550 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 rx=0x7f5dd40096a0 tx=0x7f5dd4006b40).ready entity=mgr.14156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:59:03.951 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:03.950+0000 7f5dca7fc700 1 -- 172.21.15.71:0/882079982 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+153237 (secure 0 0 0) 0x7f5dcc01cb10 con 0x7f5dd80f52d0 2024-02-20T12:59:04.155 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:04.155+0000 7f5ddecd1700 1 -- 172.21.15.71:0/882079982 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f5dd80faeb0 con 0x7f5dd80f52d0 2024-02-20T12:59:04.156 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:04.156+0000 7f5dca7fc700 1 -- 172.21.15.71:0/882079982 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7f5dcc02b020 con 0x7f5dd80f52d0 2024-02-20T12:59:04.156 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-20T12:59:04.157 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":1,"fsid":"680ee126-cfee-11ee-95bb-87774f69a715","modified":"2024-02-20T12:49:44.699059Z","created":"2024-02-20T12:49:44.699059Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi064","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:3300","nonce":0},{"type":"v1","addr":"172.21.15.64:6789","nonce":0}]},"addr":"172.21.15.64:6789/0","public_addr":"172.21.15.64:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-20T12:59:04.158 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:04.157+0000 7f5ddecd1700 1 -- 172.21.15.71:0/882079982 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f5dc4031090 msgr2=0x7f5dc4033550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:59:04.158 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:04.157+0000 7f5ddecd1700 1 --2- 172.21.15.71:0/882079982 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f5dc4031090 0x7f5dc4033550 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 rx=0x7f5dd40096a0 tx=0x7f5dd4006b40).stop 2024-02-20T12:59:04.158 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:04.157+0000 7f5ddecd1700 1 -- 172.21.15.71:0/882079982 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5dd80f52d0 msgr2=0x7f5dd8105570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:59:04.158 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:04.157+0000 7f5ddecd1700 1 --2- 172.21.15.71:0/882079982 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5dd80f52d0 0x7f5dd8105570 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 rx=0x7f5dcc0089f0 tx=0x7f5dcc00b040).stop 2024-02-20T12:59:04.158 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:04.157+0000 7f5ddecd1700 1 -- 172.21.15.71:0/882079982 shutdown_connections 2024-02-20T12:59:04.158 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:04.157+0000 7f5ddecd1700 1 --2- 172.21.15.71:0/882079982 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f5dc4031090 0x7f5dc4033550 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:59:04.159 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:04.157+0000 7f5ddecd1700 1 --2- 172.21.15.71:0/882079982 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5dd80f52d0 0x7f5dd8105570 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:59:04.159 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:04.157+0000 7f5ddecd1700 1 -- 172.21.15.71:0/882079982 >> 172.21.15.71:0/882079982 conn(0x7f5dd80f0a30 msgr2=0x7f5dd80f1560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:59:04.159 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:04.157+0000 7f5ddecd1700 1 -- 172.21.15.71:0/882079982 shutdown_connections 2024-02-20T12:59:04.159 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:04.157+0000 7f5ddecd1700 1 -- 172.21.15.71:0/882079982 wait complete. 2024-02-20T12:59:04.159 INFO:teuthology.orchestra.run.smithi071.stderr:dumped monmap epoch 1 2024-02-20T12:59:05.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:59:04 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:59:03.782782+0000 mgr.smithi064.vriyor (mgr.14156) 256 : cluster [DBG] pgmap v239: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:59:05.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:59:04 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:59:03.860773+0000 mon.smithi064 (mon.0) 253 : audit [INF] from='mgr.14156 172.21.15.64:0/3284974785' entity='mgr.smithi064.vriyor' 2024-02-20T12:59:05.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:59:04 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:59:04.156620+0000 mon.smithi064 (mon.0) 254 : audit [DBG] from='client.? 172.21.15.71:0/882079982' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-20T12:59:05.831 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-20T12:59:05.832 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/daemon-base:latest-pacific shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 680ee126-cfee-11ee-95bb-87774f69a715 -- ceph mon dump -f json 2024-02-20T12:59:07.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:59:06 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:59:05.783139+0000 mgr.smithi064.vriyor (mgr.14156) 257 : cluster [DBG] pgmap v240: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:59:07.911 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:07.910+0000 7f6aaad56700 1 -- 172.21.15.71:0/340803276 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6aa40f6590 msgr2=0x7f6aa40f69b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:59:07.911 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:07.910+0000 7f6aaad56700 1 --2- 172.21.15.71:0/340803276 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6aa40f6590 0x7f6aa40f69b0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 rx=0x7f6a94004760 tx=0x7f6a94009ad0).stop 2024-02-20T12:59:07.911 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:07.910+0000 7f6aaad56700 1 -- 172.21.15.71:0/340803276 shutdown_connections 2024-02-20T12:59:07.911 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:07.910+0000 7f6aaad56700 1 --2- 172.21.15.71:0/340803276 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6aa40f6590 0x7f6aa40f69b0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:59:07.911 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:07.910+0000 7f6aaad56700 1 -- 172.21.15.71:0/340803276 >> 172.21.15.71:0/340803276 conn(0x7f6aa40f1cd0 msgr2=0x7f6aa40f4110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:59:07.912 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:07.910+0000 7f6aaad56700 1 -- 172.21.15.71:0/340803276 shutdown_connections 2024-02-20T12:59:07.912 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:07.910+0000 7f6aaad56700 1 -- 172.21.15.71:0/340803276 wait complete. 2024-02-20T12:59:07.912 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:07.911+0000 7f6aaad56700 1 Processor -- start 2024-02-20T12:59:07.912 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:07.911+0000 7f6aaad56700 1 -- start start 2024-02-20T12:59:07.914 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:07.911+0000 7f6aaad56700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6aa40f6590 0x7f6aa4108a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:59:07.914 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:07.911+0000 7f6aaad56700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6aa4108f00 con 0x7f6aa40f6590 2024-02-20T12:59:07.914 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:07.911+0000 7f6aa8af2700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6aa40f6590 0x7f6aa4108a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:59:07.914 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:07.911+0000 7f6aa8af2700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6aa40f6590 0x7f6aa4108a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.71:38288/0 (socket says 172.21.15.71:38288) 2024-02-20T12:59:07.914 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:07.911+0000 7f6aa8af2700 1 -- 172.21.15.71:0/3411553354 learned_addr learned my addr 172.21.15.71:0/3411553354 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-02-20T12:59:07.914 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:07.912+0000 7f6aa8af2700 1 -- 172.21.15.71:0/3411553354 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a94009d70 con 0x7f6aa40f6590 2024-02-20T12:59:07.914 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:07.912+0000 7f6aa8af2700 1 --2- 172.21.15.71:0/3411553354 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6aa40f6590 0x7f6aa4108a10 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 rx=0x7f6a94004170 tx=0x7f6a94009ad0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:59:07.914 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:07.912+0000 7f6aa17fa700 1 -- 172.21.15.71:0/3411553354 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6a9400e040 con 0x7f6aa40f6590 2024-02-20T12:59:07.914 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:07.912+0000 7f6aaad56700 1 -- 172.21.15.71:0/3411553354 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6aa4109100 con 0x7f6aa40f6590 2024-02-20T12:59:07.914 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:07.912+0000 7f6aa17fa700 1 -- 172.21.15.71:0/3411553354 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(25 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f6a94008ef0 con 0x7f6aa40f6590 2024-02-20T12:59:07.914 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:07.912+0000 7f6aa17fa700 1 -- 172.21.15.71:0/3411553354 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6a94014c90 con 0x7f6aa40f6590 2024-02-20T12:59:07.914 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:07.912+0000 7f6aaad56700 1 -- 172.21.15.71:0/3411553354 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6aa4109590 con 0x7f6aa40f6590 2024-02-20T12:59:07.914 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:07.913+0000 7f6aa17fa700 1 -- 172.21.15.71:0/3411553354 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 37151+0+0 (secure 0 0 0) 0x7f6a94014420 con 0x7f6aa40f6590 2024-02-20T12:59:07.915 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:07.913+0000 7f6aaad56700 1 -- 172.21.15.71:0/3411553354 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6aa4045bb0 con 0x7f6aa40f6590 2024-02-20T12:59:07.915 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:07.913+0000 7f6aa17fa700 1 --2- 172.21.15.71:0/3411553354 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f6a8c031090 0x7f6a8c033550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:59:07.915 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:07.913+0000 7f6aa17fa700 1 -- 172.21.15.71:0/3411553354 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1065+0+0 (secure 0 0 0) 0x7f6a940422d0 con 0x7f6aa40f6590 2024-02-20T12:59:07.915 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:07.913+0000 7f6aa3fff700 1 --2- 172.21.15.71:0/3411553354 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f6a8c031090 0x7f6a8c033550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:59:07.915 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:07.914+0000 7f6aa3fff700 1 --2- 172.21.15.71:0/3411553354 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f6a8c031090 0x7f6a8c033550 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 rx=0x7f6a980096a0 tx=0x7f6a98006b40).ready entity=mgr.14156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:59:07.918 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:07.918+0000 7f6aa17fa700 1 -- 172.21.15.71:0/3411553354 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+153237 (secure 0 0 0) 0x7f6a94026300 con 0x7f6aa40f6590 2024-02-20T12:59:08.124 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:08.124+0000 7f6aaad56700 1 -- 172.21.15.71:0/3411553354 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f6aa410a390 con 0x7f6aa40f6590 2024-02-20T12:59:08.125 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:08.124+0000 7f6aa17fa700 1 -- 172.21.15.71:0/3411553354 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7f6a94012b20 con 0x7f6aa40f6590 2024-02-20T12:59:08.125 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-20T12:59:08.125 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":1,"fsid":"680ee126-cfee-11ee-95bb-87774f69a715","modified":"2024-02-20T12:49:44.699059Z","created":"2024-02-20T12:49:44.699059Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi064","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:3300","nonce":0},{"type":"v1","addr":"172.21.15.64:6789","nonce":0}]},"addr":"172.21.15.64:6789/0","public_addr":"172.21.15.64:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-20T12:59:08.126 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:08.126+0000 7f6aaad56700 1 -- 172.21.15.71:0/3411553354 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f6a8c031090 msgr2=0x7f6a8c033550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:59:08.127 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:08.126+0000 7f6aaad56700 1 --2- 172.21.15.71:0/3411553354 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f6a8c031090 0x7f6a8c033550 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 rx=0x7f6a980096a0 tx=0x7f6a98006b40).stop 2024-02-20T12:59:08.127 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:08.126+0000 7f6aaad56700 1 -- 172.21.15.71:0/3411553354 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6aa40f6590 msgr2=0x7f6aa4108a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:59:08.127 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:08.126+0000 7f6aaad56700 1 --2- 172.21.15.71:0/3411553354 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6aa40f6590 0x7f6aa4108a10 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 rx=0x7f6a94004170 tx=0x7f6a94009ad0).stop 2024-02-20T12:59:08.127 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:08.126+0000 7f6aaad56700 1 -- 172.21.15.71:0/3411553354 shutdown_connections 2024-02-20T12:59:08.127 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:08.126+0000 7f6aaad56700 1 --2- 172.21.15.71:0/3411553354 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f6a8c031090 0x7f6a8c033550 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:59:08.127 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:08.126+0000 7f6aaad56700 1 --2- 172.21.15.71:0/3411553354 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6aa40f6590 0x7f6aa4108a10 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:59:08.127 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:08.126+0000 7f6aaad56700 1 -- 172.21.15.71:0/3411553354 >> 172.21.15.71:0/3411553354 conn(0x7f6aa40f1cd0 msgr2=0x7f6aa40f2780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:59:08.127 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:08.126+0000 7f6aaad56700 1 -- 172.21.15.71:0/3411553354 shutdown_connections 2024-02-20T12:59:08.127 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:08.126+0000 7f6aaad56700 1 -- 172.21.15.71:0/3411553354 wait complete. 2024-02-20T12:59:08.127 INFO:teuthology.orchestra.run.smithi071.stderr:dumped monmap epoch 1 2024-02-20T12:59:09.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:59:08 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:59:07.783450+0000 mgr.smithi064.vriyor (mgr.14156) 258 : cluster [DBG] pgmap v241: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:59:09.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:59:08 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:59:08.125270+0000 mon.smithi064 (mon.0) 255 : audit [DBG] from='client.? 172.21.15.71:0/3411553354' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-20T12:59:09.840 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-20T12:59:09.841 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/daemon-base:latest-pacific shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 680ee126-cfee-11ee-95bb-87774f69a715 -- ceph mon dump -f json 2024-02-20T12:59:11.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:59:10 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:59:09.783693+0000 mgr.smithi064.vriyor (mgr.14156) 259 : cluster [DBG] pgmap v242: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:59:11.898 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:11.857+0000 7f4f03cb8700 1 -- 172.21.15.71:0/765402590 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4efc060360 msgr2=0x7f4efc0f9160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:59:11.898 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:11.857+0000 7f4f03cb8700 1 --2- 172.21.15.71:0/765402590 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4efc060360 0x7f4efc0f9160 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 rx=0x7f4eec009fb0 tx=0x7f4eec009b30).stop 2024-02-20T12:59:11.898 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:11.857+0000 7f4f03cb8700 1 -- 172.21.15.71:0/765402590 shutdown_connections 2024-02-20T12:59:11.898 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:11.857+0000 7f4f03cb8700 1 --2- 172.21.15.71:0/765402590 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4efc060360 0x7f4efc0f9160 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:59:11.898 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:11.857+0000 7f4f03cb8700 1 -- 172.21.15.71:0/765402590 >> 172.21.15.71:0/765402590 conn(0x7f4efc0f0be0 msgr2=0x7f4efc0f3000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:59:11.898 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:11.857+0000 7f4f03cb8700 1 -- 172.21.15.71:0/765402590 shutdown_connections 2024-02-20T12:59:11.898 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:11.857+0000 7f4f03cb8700 1 -- 172.21.15.71:0/765402590 wait complete. 2024-02-20T12:59:11.898 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:11.858+0000 7f4f03cb8700 1 Processor -- start 2024-02-20T12:59:11.898 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:11.858+0000 7f4f03cb8700 1 -- start start 2024-02-20T12:59:11.899 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:11.858+0000 7f4f03cb8700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4efc060360 0x7f4efc106750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:59:11.899 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:11.858+0000 7f4f03cb8700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4efc106c40 con 0x7f4efc060360 2024-02-20T12:59:11.899 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:11.858+0000 7f4f01a54700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4efc060360 0x7f4efc106750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:59:11.899 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:11.858+0000 7f4f01a54700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4efc060360 0x7f4efc106750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.71:38300/0 (socket says 172.21.15.71:38300) 2024-02-20T12:59:11.899 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:11.858+0000 7f4f01a54700 1 -- 172.21.15.71:0/4016347605 learned_addr learned my addr 172.21.15.71:0/4016347605 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-02-20T12:59:11.899 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:11.858+0000 7f4f01a54700 1 -- 172.21.15.71:0/4016347605 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4eec005040 con 0x7f4efc060360 2024-02-20T12:59:11.899 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:11.858+0000 7f4f01a54700 1 --2- 172.21.15.71:0/4016347605 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4efc060360 0x7f4efc106750 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 rx=0x7f4eec009f80 tx=0x7f4eec00b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:59:11.899 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:11.859+0000 7f4ef27fc700 1 -- 172.21.15.71:0/4016347605 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4eec009320 con 0x7f4efc060360 2024-02-20T12:59:11.899 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:11.859+0000 7f4f03cb8700 1 -- 172.21.15.71:0/4016347605 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4efc106e40 con 0x7f4efc060360 2024-02-20T12:59:11.899 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:11.859+0000 7f4ef27fc700 1 -- 172.21.15.71:0/4016347605 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(25 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f4eec013680 con 0x7f4efc060360 2024-02-20T12:59:11.899 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:11.859+0000 7f4ef27fc700 1 -- 172.21.15.71:0/4016347605 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4eec01eda0 con 0x7f4efc060360 2024-02-20T12:59:11.899 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:11.859+0000 7f4f03cb8700 1 -- 172.21.15.71:0/4016347605 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4efc1072d0 con 0x7f4efc060360 2024-02-20T12:59:11.899 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:11.859+0000 7f4ef27fc700 1 -- 172.21.15.71:0/4016347605 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 37151+0+0 (secure 0 0 0) 0x7f4eec01e420 con 0x7f4efc060360 2024-02-20T12:59:11.899 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:11.860+0000 7f4f03cb8700 1 -- 172.21.15.71:0/4016347605 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4efc045bb0 con 0x7f4efc060360 2024-02-20T12:59:11.900 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:11.860+0000 7f4ef27fc700 1 --2- 172.21.15.71:0/4016347605 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f4ee8031090 0x7f4ee8033550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:59:11.900 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:11.860+0000 7f4ef27fc700 1 -- 172.21.15.71:0/4016347605 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1065+0+0 (secure 0 0 0) 0x7f4eec042470 con 0x7f4efc060360 2024-02-20T12:59:11.900 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:11.860+0000 7f4f01253700 1 --2- 172.21.15.71:0/4016347605 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f4ee8031090 0x7f4ee8033550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:59:11.900 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:11.860+0000 7f4f01253700 1 --2- 172.21.15.71:0/4016347605 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f4ee8031090 0x7f4ee8033550 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 rx=0x7f4ef80096a0 tx=0x7f4ef8006b40).ready entity=mgr.14156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:59:11.900 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:11.865+0000 7f4ef27fc700 1 -- 172.21.15.71:0/4016347605 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+153237 (secure 0 0 0) 0x7f4eec01d380 con 0x7f4efc060360 2024-02-20T12:59:12.075 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:12.074+0000 7f4f03cb8700 1 -- 172.21.15.71:0/4016347605 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f4efc108180 con 0x7f4efc060360 2024-02-20T12:59:12.075 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:12.075+0000 7f4ef27fc700 1 -- 172.21.15.71:0/4016347605 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7f4eec004490 con 0x7f4efc060360 2024-02-20T12:59:12.076 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-20T12:59:12.076 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":1,"fsid":"680ee126-cfee-11ee-95bb-87774f69a715","modified":"2024-02-20T12:49:44.699059Z","created":"2024-02-20T12:49:44.699059Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi064","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:3300","nonce":0},{"type":"v1","addr":"172.21.15.64:6789","nonce":0}]},"addr":"172.21.15.64:6789/0","public_addr":"172.21.15.64:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-20T12:59:12.077 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:12.076+0000 7f4f03cb8700 1 -- 172.21.15.71:0/4016347605 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f4ee8031090 msgr2=0x7f4ee8033550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:59:12.078 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:12.077+0000 7f4f03cb8700 1 --2- 172.21.15.71:0/4016347605 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f4ee8031090 0x7f4ee8033550 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 rx=0x7f4ef80096a0 tx=0x7f4ef8006b40).stop 2024-02-20T12:59:12.078 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:12.077+0000 7f4f03cb8700 1 -- 172.21.15.71:0/4016347605 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4efc060360 msgr2=0x7f4efc106750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:59:12.078 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:12.077+0000 7f4f03cb8700 1 --2- 172.21.15.71:0/4016347605 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4efc060360 0x7f4efc106750 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 rx=0x7f4eec009f80 tx=0x7f4eec00b040).stop 2024-02-20T12:59:12.078 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:12.077+0000 7f4f03cb8700 1 -- 172.21.15.71:0/4016347605 shutdown_connections 2024-02-20T12:59:12.078 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:12.077+0000 7f4f03cb8700 1 --2- 172.21.15.71:0/4016347605 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f4ee8031090 0x7f4ee8033550 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:59:12.078 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:12.077+0000 7f4f03cb8700 1 --2- 172.21.15.71:0/4016347605 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4efc060360 0x7f4efc106750 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:59:12.078 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:12.077+0000 7f4f03cb8700 1 -- 172.21.15.71:0/4016347605 >> 172.21.15.71:0/4016347605 conn(0x7f4efc0f0be0 msgr2=0x7f4efc0f1660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:59:12.078 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:12.077+0000 7f4f03cb8700 1 -- 172.21.15.71:0/4016347605 shutdown_connections 2024-02-20T12:59:12.079 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:12.077+0000 7f4f03cb8700 1 -- 172.21.15.71:0/4016347605 wait complete. 2024-02-20T12:59:12.079 INFO:teuthology.orchestra.run.smithi071.stderr:dumped monmap epoch 1 2024-02-20T12:59:13.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:59:12 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:59:11.784155+0000 mgr.smithi064.vriyor (mgr.14156) 260 : cluster [DBG] pgmap v243: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:59:13.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:59:12 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:59:12.075800+0000 mon.smithi064 (mon.0) 256 : audit [DBG] from='client.? 172.21.15.71:0/4016347605' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-20T12:59:13.758 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-20T12:59:13.759 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/daemon-base:latest-pacific shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 680ee126-cfee-11ee-95bb-87774f69a715 -- ceph mon dump -f json 2024-02-20T12:59:15.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:59:14 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:59:13.784668+0000 mgr.smithi064.vriyor (mgr.14156) 261 : cluster [DBG] pgmap v244: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:59:15.814 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:15.813+0000 7f3d6c547700 1 -- 172.21.15.71:0/1168014887 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3d640f6590 msgr2=0x7f3d640f69b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:59:15.814 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:15.813+0000 7f3d6c547700 1 --2- 172.21.15.71:0/1168014887 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3d640f6590 0x7f3d640f69b0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 rx=0x7f3d54004760 tx=0x7f3d54009ad0).stop 2024-02-20T12:59:15.814 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:15.813+0000 7f3d6c547700 1 -- 172.21.15.71:0/1168014887 shutdown_connections 2024-02-20T12:59:15.814 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:15.813+0000 7f3d6c547700 1 --2- 172.21.15.71:0/1168014887 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3d640f6590 0x7f3d640f69b0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:59:15.814 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:15.813+0000 7f3d6c547700 1 -- 172.21.15.71:0/1168014887 >> 172.21.15.71:0/1168014887 conn(0x7f3d640f1cd0 msgr2=0x7f3d640f4110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:59:15.814 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:15.813+0000 7f3d6c547700 1 -- 172.21.15.71:0/1168014887 shutdown_connections 2024-02-20T12:59:15.814 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:15.813+0000 7f3d6c547700 1 -- 172.21.15.71:0/1168014887 wait complete. 2024-02-20T12:59:15.814 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:15.813+0000 7f3d6c547700 1 Processor -- start 2024-02-20T12:59:15.815 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:15.813+0000 7f3d6c547700 1 -- start start 2024-02-20T12:59:15.815 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:15.813+0000 7f3d6c547700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3d640f6590 0x7f3d6406ba80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:59:15.815 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:15.813+0000 7f3d6c547700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d64069f00 con 0x7f3d640f6590 2024-02-20T12:59:15.815 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:15.814+0000 7f3d6a2e3700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3d640f6590 0x7f3d6406ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:59:15.815 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:15.814+0000 7f3d6a2e3700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3d640f6590 0x7f3d6406ba80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.71:38320/0 (socket says 172.21.15.71:38320) 2024-02-20T12:59:15.815 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:15.814+0000 7f3d6a2e3700 1 -- 172.21.15.71:0/3765428672 learned_addr learned my addr 172.21.15.71:0/3765428672 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-02-20T12:59:15.817 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:15.814+0000 7f3d6a2e3700 1 -- 172.21.15.71:0/3765428672 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d54009d70 con 0x7f3d640f6590 2024-02-20T12:59:15.817 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:15.814+0000 7f3d6a2e3700 1 --2- 172.21.15.71:0/3765428672 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3d640f6590 0x7f3d6406ba80 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 rx=0x7f3d54004170 tx=0x7f3d54009ad0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:59:15.817 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:15.814+0000 7f3d5affd700 1 -- 172.21.15.71:0/3765428672 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3d5400e040 con 0x7f3d640f6590 2024-02-20T12:59:15.817 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:15.814+0000 7f3d5affd700 1 -- 172.21.15.71:0/3765428672 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(25 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f3d54008ef0 con 0x7f3d640f6590 2024-02-20T12:59:15.817 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:15.814+0000 7f3d6c547700 1 -- 172.21.15.71:0/3765428672 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d6406a0a0 con 0x7f3d640f6590 2024-02-20T12:59:15.817 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:15.814+0000 7f3d5affd700 1 -- 172.21.15.71:0/3765428672 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3d54014c90 con 0x7f3d640f6590 2024-02-20T12:59:15.817 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:15.814+0000 7f3d6c547700 1 -- 172.21.15.71:0/3765428672 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d6406a530 con 0x7f3d640f6590 2024-02-20T12:59:15.817 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:15.815+0000 7f3d5affd700 1 -- 172.21.15.71:0/3765428672 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 37151+0+0 (secure 0 0 0) 0x7f3d54014420 con 0x7f3d640f6590 2024-02-20T12:59:15.817 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:15.815+0000 7f3d6c547700 1 -- 172.21.15.71:0/3765428672 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d640591e0 con 0x7f3d640f6590 2024-02-20T12:59:15.817 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:15.815+0000 7f3d5affd700 1 --2- 172.21.15.71:0/3765428672 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f3d50031090 0x7f3d50033550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:59:15.817 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:15.816+0000 7f3d5affd700 1 -- 172.21.15.71:0/3765428672 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1065+0+0 (secure 0 0 0) 0x7f3d54042240 con 0x7f3d640f6590 2024-02-20T12:59:15.817 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:15.816+0000 7f3d69ae2700 1 --2- 172.21.15.71:0/3765428672 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f3d50031090 0x7f3d50033550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:59:15.817 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:15.816+0000 7f3d69ae2700 1 --2- 172.21.15.71:0/3765428672 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f3d50031090 0x7f3d50033550 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 rx=0x7f3d600096a0 tx=0x7f3d60006b40).ready entity=mgr.14156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:59:15.820 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:15.820+0000 7f3d5affd700 1 -- 172.21.15.71:0/3765428672 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+153237 (secure 0 0 0) 0x7f3d54026300 con 0x7f3d640f6590 2024-02-20T12:59:16.030 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:16.029+0000 7f3d6c547700 1 -- 172.21.15.71:0/3765428672 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f3d640fc080 con 0x7f3d640f6590 2024-02-20T12:59:16.030 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:16.030+0000 7f3d5affd700 1 -- 172.21.15.71:0/3765428672 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7f3d54012b20 con 0x7f3d640f6590 2024-02-20T12:59:16.031 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-20T12:59:16.031 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":1,"fsid":"680ee126-cfee-11ee-95bb-87774f69a715","modified":"2024-02-20T12:49:44.699059Z","created":"2024-02-20T12:49:44.699059Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi064","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:3300","nonce":0},{"type":"v1","addr":"172.21.15.64:6789","nonce":0}]},"addr":"172.21.15.64:6789/0","public_addr":"172.21.15.64:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-20T12:59:16.032 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:16.031+0000 7f3d6c547700 1 -- 172.21.15.71:0/3765428672 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f3d50031090 msgr2=0x7f3d50033550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:59:16.032 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:16.031+0000 7f3d6c547700 1 --2- 172.21.15.71:0/3765428672 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f3d50031090 0x7f3d50033550 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 rx=0x7f3d600096a0 tx=0x7f3d60006b40).stop 2024-02-20T12:59:16.033 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:16.032+0000 7f3d6c547700 1 -- 172.21.15.71:0/3765428672 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3d640f6590 msgr2=0x7f3d6406ba80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:59:16.033 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:16.032+0000 7f3d6c547700 1 --2- 172.21.15.71:0/3765428672 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3d640f6590 0x7f3d6406ba80 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 rx=0x7f3d54004170 tx=0x7f3d54009ad0).stop 2024-02-20T12:59:16.033 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:16.032+0000 7f3d6c547700 1 -- 172.21.15.71:0/3765428672 shutdown_connections 2024-02-20T12:59:16.033 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:16.032+0000 7f3d6c547700 1 --2- 172.21.15.71:0/3765428672 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f3d50031090 0x7f3d50033550 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:59:16.033 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:16.032+0000 7f3d6c547700 1 --2- 172.21.15.71:0/3765428672 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3d640f6590 0x7f3d6406ba80 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:59:16.033 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:16.032+0000 7f3d6c547700 1 -- 172.21.15.71:0/3765428672 >> 172.21.15.71:0/3765428672 conn(0x7f3d640f1cd0 msgr2=0x7f3d640f27c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:59:16.033 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:16.032+0000 7f3d6c547700 1 -- 172.21.15.71:0/3765428672 shutdown_connections 2024-02-20T12:59:16.033 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:16.032+0000 7f3d6c547700 1 -- 172.21.15.71:0/3765428672 wait complete. 2024-02-20T12:59:16.033 INFO:teuthology.orchestra.run.smithi071.stderr:dumped monmap epoch 1 2024-02-20T12:59:17.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:59:16 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:59:15.785059+0000 mgr.smithi064.vriyor (mgr.14156) 262 : cluster [DBG] pgmap v245: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:59:17.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:59:16 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:59:16.030805+0000 mon.smithi064 (mon.0) 257 : audit [DBG] from='client.? 172.21.15.71:0/3765428672' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-20T12:59:17.677 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-20T12:59:17.678 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/daemon-base:latest-pacific shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 680ee126-cfee-11ee-95bb-87774f69a715 -- ceph mon dump -f json 2024-02-20T12:59:19.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:59:18 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:59:17.785432+0000 mgr.smithi064.vriyor (mgr.14156) 263 : cluster [DBG] pgmap v246: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:59:19.698 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:19.695+0000 7f929ec03700 1 -- 172.21.15.71:0/2217463994 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f92980f63e0 msgr2=0x7f92980f6800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:59:19.699 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:19.695+0000 7f929ec03700 1 --2- 172.21.15.71:0/2217463994 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f92980f63e0 0x7f92980f6800 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 rx=0x7f9294009fb0 tx=0x7f9294009b30).stop 2024-02-20T12:59:19.699 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:19.695+0000 7f929ec03700 1 -- 172.21.15.71:0/2217463994 shutdown_connections 2024-02-20T12:59:19.699 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:19.695+0000 7f929ec03700 1 --2- 172.21.15.71:0/2217463994 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f92980f63e0 0x7f92980f6800 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:59:19.699 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:19.695+0000 7f929ec03700 1 -- 172.21.15.71:0/2217463994 >> 172.21.15.71:0/2217463994 conn(0x7f92980f1b40 msgr2=0x7f92980f3f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:59:19.699 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:19.695+0000 7f929ec03700 1 -- 172.21.15.71:0/2217463994 shutdown_connections 2024-02-20T12:59:19.699 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:19.695+0000 7f929ec03700 1 -- 172.21.15.71:0/2217463994 wait complete. 2024-02-20T12:59:19.699 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:19.696+0000 7f929ec03700 1 Processor -- start 2024-02-20T12:59:19.699 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:19.696+0000 7f929ec03700 1 -- start start 2024-02-20T12:59:19.699 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:19.696+0000 7f929ec03700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f92980f63e0 0x7f9298106620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:59:19.699 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:19.696+0000 7f929ec03700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9298106b10 con 0x7f92980f63e0 2024-02-20T12:59:19.699 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:19.696+0000 7f929dc01700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f92980f63e0 0x7f9298106620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:59:19.699 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:19.696+0000 7f929dc01700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f92980f63e0 0x7f9298106620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.71:52662/0 (socket says 172.21.15.71:52662) 2024-02-20T12:59:19.700 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:19.696+0000 7f929dc01700 1 -- 172.21.15.71:0/2428899756 learned_addr learned my addr 172.21.15.71:0/2428899756 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-02-20T12:59:19.700 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:19.697+0000 7f929dc01700 1 -- 172.21.15.71:0/2428899756 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9294005040 con 0x7f92980f63e0 2024-02-20T12:59:19.700 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:19.697+0000 7f929dc01700 1 --2- 172.21.15.71:0/2428899756 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f92980f63e0 0x7f9298106620 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 rx=0x7f92940089f0 tx=0x7f929400b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:59:19.700 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:19.697+0000 7f928e7fc700 1 -- 172.21.15.71:0/2428899756 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9294009320 con 0x7f92980f63e0 2024-02-20T12:59:19.700 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:19.697+0000 7f928e7fc700 1 -- 172.21.15.71:0/2428899756 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(25 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f9294013ec0 con 0x7f92980f63e0 2024-02-20T12:59:19.700 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:19.697+0000 7f929ec03700 1 -- 172.21.15.71:0/2428899756 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9298106d10 con 0x7f92980f63e0 2024-02-20T12:59:19.700 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:19.697+0000 7f928e7fc700 1 -- 172.21.15.71:0/2428899756 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9294014d30 con 0x7f92980f63e0 2024-02-20T12:59:19.700 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:19.697+0000 7f929ec03700 1 -- 172.21.15.71:0/2428899756 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92981071a0 con 0x7f92980f63e0 2024-02-20T12:59:19.701 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:19.698+0000 7f928e7fc700 1 -- 172.21.15.71:0/2428899756 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 37151+0+0 (secure 0 0 0) 0x7f92940263a0 con 0x7f92980f63e0 2024-02-20T12:59:19.701 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:19.698+0000 7f928e7fc700 1 --2- 172.21.15.71:0/2428899756 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f9284031090 0x7f9284033550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:59:19.701 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:19.698+0000 7f928e7fc700 1 -- 172.21.15.71:0/2428899756 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1065+0+0 (secure 0 0 0) 0x7f92940425a0 con 0x7f92980f63e0 2024-02-20T12:59:19.701 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:19.698+0000 7f929ec03700 1 -- 172.21.15.71:0/2428899756 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92980591e0 con 0x7f92980f63e0 2024-02-20T12:59:19.701 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:19.698+0000 7f929d400700 1 --2- 172.21.15.71:0/2428899756 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f9284031090 0x7f9284033550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:59:19.701 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:19.699+0000 7f929d400700 1 --2- 172.21.15.71:0/2428899756 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f9284031090 0x7f9284033550 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 rx=0x7f92880096a0 tx=0x7f9288006b40).ready entity=mgr.14156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:59:19.704 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:19.703+0000 7f928e7fc700 1 -- 172.21.15.71:0/2428899756 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+153237 (secure 0 0 0) 0x7f92940266d0 con 0x7f92980f63e0 2024-02-20T12:59:19.914 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:19.912+0000 7f929ec03700 1 -- 172.21.15.71:0/2428899756 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f9298045bb0 con 0x7f92980f63e0 2024-02-20T12:59:19.915 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:19.913+0000 7f928e7fc700 1 -- 172.21.15.71:0/2428899756 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7f929402b020 con 0x7f92980f63e0 2024-02-20T12:59:19.915 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-20T12:59:19.915 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":1,"fsid":"680ee126-cfee-11ee-95bb-87774f69a715","modified":"2024-02-20T12:49:44.699059Z","created":"2024-02-20T12:49:44.699059Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi064","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:3300","nonce":0},{"type":"v1","addr":"172.21.15.64:6789","nonce":0}]},"addr":"172.21.15.64:6789/0","public_addr":"172.21.15.64:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-20T12:59:19.916 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:19.915+0000 7f929ec03700 1 -- 172.21.15.71:0/2428899756 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f9284031090 msgr2=0x7f9284033550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:59:19.917 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:19.915+0000 7f929ec03700 1 --2- 172.21.15.71:0/2428899756 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f9284031090 0x7f9284033550 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 rx=0x7f92880096a0 tx=0x7f9288006b40).stop 2024-02-20T12:59:19.917 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:19.915+0000 7f929ec03700 1 -- 172.21.15.71:0/2428899756 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f92980f63e0 msgr2=0x7f9298106620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:59:19.917 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:19.915+0000 7f929ec03700 1 --2- 172.21.15.71:0/2428899756 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f92980f63e0 0x7f9298106620 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 rx=0x7f92940089f0 tx=0x7f929400b040).stop 2024-02-20T12:59:19.917 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:19.915+0000 7f929ec03700 1 -- 172.21.15.71:0/2428899756 shutdown_connections 2024-02-20T12:59:19.917 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:19.915+0000 7f929ec03700 1 --2- 172.21.15.71:0/2428899756 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f9284031090 0x7f9284033550 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:59:19.917 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:19.915+0000 7f929ec03700 1 --2- 172.21.15.71:0/2428899756 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f92980f63e0 0x7f9298106620 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:59:19.917 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:19.915+0000 7f929ec03700 1 -- 172.21.15.71:0/2428899756 >> 172.21.15.71:0/2428899756 conn(0x7f92980f1b40 msgr2=0x7f92980f2660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:59:19.917 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:19.915+0000 7f929ec03700 1 -- 172.21.15.71:0/2428899756 shutdown_connections 2024-02-20T12:59:19.917 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:19.915+0000 7f929ec03700 1 -- 172.21.15.71:0/2428899756 wait complete. 2024-02-20T12:59:19.917 INFO:teuthology.orchestra.run.smithi071.stderr:dumped monmap epoch 1 2024-02-20T12:59:20.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:59:19 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:59:19.915052+0000 mon.smithi064 (mon.0) 258 : audit [DBG] from='client.? 172.21.15.71:0/2428899756' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-20T12:59:21.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:59:20 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:59:19.785870+0000 mgr.smithi064.vriyor (mgr.14156) 264 : cluster [DBG] pgmap v247: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:59:21.691 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-20T12:59:21.691 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/daemon-base:latest-pacific shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 680ee126-cfee-11ee-95bb-87774f69a715 -- ceph mon dump -f json 2024-02-20T12:59:23.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:59:22 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:59:21.786421+0000 mgr.smithi064.vriyor (mgr.14156) 265 : cluster [DBG] pgmap v248: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:59:23.678 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:23.674+0000 7f43a385b700 1 -- 172.21.15.71:0/3736955074 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f439c0f6580 msgr2=0x7f439c0f69a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:59:23.678 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:23.674+0000 7f43a385b700 1 --2- 172.21.15.71:0/3736955074 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f439c0f6580 0x7f439c0f69a0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 rx=0x7f438c009fb0 tx=0x7f438c009b30).stop 2024-02-20T12:59:23.678 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:23.674+0000 7f43a385b700 1 -- 172.21.15.71:0/3736955074 shutdown_connections 2024-02-20T12:59:23.678 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:23.674+0000 7f43a385b700 1 --2- 172.21.15.71:0/3736955074 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f439c0f6580 0x7f439c0f69a0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:59:23.678 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:23.674+0000 7f43a385b700 1 -- 172.21.15.71:0/3736955074 >> 172.21.15.71:0/3736955074 conn(0x7f439c0f1ce0 msgr2=0x7f439c0f4100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:59:23.679 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:23.675+0000 7f43a385b700 1 -- 172.21.15.71:0/3736955074 shutdown_connections 2024-02-20T12:59:23.679 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:23.675+0000 7f43a385b700 1 -- 172.21.15.71:0/3736955074 wait complete. 2024-02-20T12:59:23.679 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:23.675+0000 7f43a385b700 1 Processor -- start 2024-02-20T12:59:23.679 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:23.675+0000 7f43a385b700 1 -- start start 2024-02-20T12:59:23.679 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:23.675+0000 7f43a385b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f439c0f6580 0x7f439c1067b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:59:23.679 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:23.675+0000 7f43a385b700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f439c106ca0 con 0x7f439c0f6580 2024-02-20T12:59:23.679 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:23.675+0000 7f43a15f7700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f439c0f6580 0x7f439c1067b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:59:23.679 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:23.675+0000 7f43a15f7700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f439c0f6580 0x7f439c1067b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.71:52684/0 (socket says 172.21.15.71:52684) 2024-02-20T12:59:23.679 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:23.675+0000 7f43a15f7700 1 -- 172.21.15.71:0/3545217125 learned_addr learned my addr 172.21.15.71:0/3545217125 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-02-20T12:59:23.679 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:23.676+0000 7f43a15f7700 1 -- 172.21.15.71:0/3545217125 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f438c005040 con 0x7f439c0f6580 2024-02-20T12:59:23.679 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:23.676+0000 7f43a15f7700 1 --2- 172.21.15.71:0/3545217125 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f439c0f6580 0x7f439c1067b0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 rx=0x7f438c0089f0 tx=0x7f438c00b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:59:23.679 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:23.676+0000 7f4391ffb700 1 -- 172.21.15.71:0/3545217125 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f438c009320 con 0x7f439c0f6580 2024-02-20T12:59:23.680 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:23.676+0000 7f43a385b700 1 -- 172.21.15.71:0/3545217125 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f439c106ea0 con 0x7f439c0f6580 2024-02-20T12:59:23.680 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:23.676+0000 7f4391ffb700 1 -- 172.21.15.71:0/3545217125 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(25 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f438c013ec0 con 0x7f439c0f6580 2024-02-20T12:59:23.680 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:23.676+0000 7f4391ffb700 1 -- 172.21.15.71:0/3545217125 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f438c014d30 con 0x7f439c0f6580 2024-02-20T12:59:23.680 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:23.676+0000 7f43a385b700 1 -- 172.21.15.71:0/3545217125 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f439c107330 con 0x7f439c0f6580 2024-02-20T12:59:23.681 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:23.677+0000 7f4391ffb700 1 -- 172.21.15.71:0/3545217125 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 37151+0+0 (secure 0 0 0) 0x7f438c0095f0 con 0x7f439c0f6580 2024-02-20T12:59:23.681 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:23.677+0000 7f43a385b700 1 -- 172.21.15.71:0/3545217125 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f439c045bb0 con 0x7f439c0f6580 2024-02-20T12:59:23.681 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:23.677+0000 7f4391ffb700 1 --2- 172.21.15.71:0/3545217125 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f43880310e0 0x7f43880335a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:59:23.681 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:23.677+0000 7f4391ffb700 1 -- 172.21.15.71:0/3545217125 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1065+0+0 (secure 0 0 0) 0x7f438c0423a0 con 0x7f439c0f6580 2024-02-20T12:59:23.681 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:23.678+0000 7f43a0df6700 1 --2- 172.21.15.71:0/3545217125 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f43880310e0 0x7f43880335a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:59:23.681 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:23.678+0000 7f43a0df6700 1 --2- 172.21.15.71:0/3545217125 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f43880310e0 0x7f43880335a0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 rx=0x7f43980096a0 tx=0x7f4398006b40).ready entity=mgr.14156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:59:23.683 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:23.682+0000 7f4391ffb700 1 -- 172.21.15.71:0/3545217125 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+153237 (secure 0 0 0) 0x7f438c01d7f0 con 0x7f439c0f6580 2024-02-20T12:59:23.888 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:23.886+0000 7f43a385b700 1 -- 172.21.15.71:0/3545217125 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f439c1081e0 con 0x7f439c0f6580 2024-02-20T12:59:23.888 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:23.887+0000 7f4391ffb700 1 -- 172.21.15.71:0/3545217125 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7f438c00fe40 con 0x7f439c0f6580 2024-02-20T12:59:23.889 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-20T12:59:23.889 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":1,"fsid":"680ee126-cfee-11ee-95bb-87774f69a715","modified":"2024-02-20T12:49:44.699059Z","created":"2024-02-20T12:49:44.699059Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi064","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:3300","nonce":0},{"type":"v1","addr":"172.21.15.64:6789","nonce":0}]},"addr":"172.21.15.64:6789/0","public_addr":"172.21.15.64:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-20T12:59:23.890 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:23.889+0000 7f43a385b700 1 -- 172.21.15.71:0/3545217125 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f43880310e0 msgr2=0x7f43880335a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:59:23.890 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:23.889+0000 7f43a385b700 1 --2- 172.21.15.71:0/3545217125 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f43880310e0 0x7f43880335a0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 rx=0x7f43980096a0 tx=0x7f4398006b40).stop 2024-02-20T12:59:23.890 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:23.889+0000 7f43a385b700 1 -- 172.21.15.71:0/3545217125 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f439c0f6580 msgr2=0x7f439c1067b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:59:23.890 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:23.889+0000 7f43a385b700 1 --2- 172.21.15.71:0/3545217125 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f439c0f6580 0x7f439c1067b0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 rx=0x7f438c0089f0 tx=0x7f438c00b040).stop 2024-02-20T12:59:23.891 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:23.889+0000 7f43a385b700 1 -- 172.21.15.71:0/3545217125 shutdown_connections 2024-02-20T12:59:23.891 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:23.889+0000 7f43a385b700 1 --2- 172.21.15.71:0/3545217125 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f43880310e0 0x7f43880335a0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:59:23.891 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:23.889+0000 7f43a385b700 1 --2- 172.21.15.71:0/3545217125 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f439c0f6580 0x7f439c1067b0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:59:23.891 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:23.889+0000 7f43a385b700 1 -- 172.21.15.71:0/3545217125 >> 172.21.15.71:0/3545217125 conn(0x7f439c0f1ce0 msgr2=0x7f439c0f2800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:59:23.891 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:23.889+0000 7f43a385b700 1 -- 172.21.15.71:0/3545217125 shutdown_connections 2024-02-20T12:59:23.891 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:23.889+0000 7f43a385b700 1 -- 172.21.15.71:0/3545217125 wait complete. 2024-02-20T12:59:23.891 INFO:teuthology.orchestra.run.smithi071.stderr:dumped monmap epoch 1 2024-02-20T12:59:24.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:59:23 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:59:23.889038+0000 mon.smithi064 (mon.0) 259 : audit [DBG] from='client.? 172.21.15.71:0/3545217125' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-20T12:59:25.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:59:24 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:59:23.786921+0000 mgr.smithi064.vriyor (mgr.14156) 266 : cluster [DBG] pgmap v249: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:59:25.592 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-20T12:59:25.592 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/daemon-base:latest-pacific shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 680ee126-cfee-11ee-95bb-87774f69a715 -- ceph mon dump -f json 2024-02-20T12:59:27.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:59:26 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:59:25.787287+0000 mgr.smithi064.vriyor (mgr.14156) 267 : cluster [DBG] pgmap v250: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:59:27.674 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:27.631+0000 7ff100812700 1 -- 172.21.15.71:0/3513421275 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff0fc060360 msgr2=0x7ff0fc0f8fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:59:27.675 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:27.631+0000 7ff100812700 1 --2- 172.21.15.71:0/3513421275 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff0fc060360 0x7ff0fc0f8fb0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 rx=0x7ff0ec009960 tx=0x7ff0ec004d10).stop 2024-02-20T12:59:27.675 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:27.631+0000 7ff100812700 1 -- 172.21.15.71:0/3513421275 shutdown_connections 2024-02-20T12:59:27.675 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:27.631+0000 7ff100812700 1 --2- 172.21.15.71:0/3513421275 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff0fc060360 0x7ff0fc0f8fb0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:59:27.675 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:27.631+0000 7ff100812700 1 -- 172.21.15.71:0/3513421275 >> 172.21.15.71:0/3513421275 conn(0x7ff0fc0f0a30 msgr2=0x7ff0fc0f2e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:59:27.675 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:27.631+0000 7ff100812700 1 -- 172.21.15.71:0/3513421275 shutdown_connections 2024-02-20T12:59:27.675 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:27.631+0000 7ff100812700 1 -- 172.21.15.71:0/3513421275 wait complete. 2024-02-20T12:59:27.675 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:27.632+0000 7ff100812700 1 Processor -- start 2024-02-20T12:59:27.675 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:27.632+0000 7ff100812700 1 -- start start 2024-02-20T12:59:27.675 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:27.632+0000 7ff100812700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff0fc060360 0x7ff0fc0f6a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:59:27.675 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:27.632+0000 7ff100812700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0fc0f4eb0 con 0x7ff0fc060360 2024-02-20T12:59:27.675 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:27.632+0000 7ff0fb7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff0fc060360 0x7ff0fc0f6a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:59:27.676 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:27.632+0000 7ff0fb7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff0fc060360 0x7ff0fc0f6a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.71:56266/0 (socket says 172.21.15.71:56266) 2024-02-20T12:59:27.676 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:27.632+0000 7ff0fb7fe700 1 -- 172.21.15.71:0/2336784255 learned_addr learned my addr 172.21.15.71:0/2336784255 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-02-20T12:59:27.676 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:27.633+0000 7ff0fb7fe700 1 -- 172.21.15.71:0/2336784255 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0ec009640 con 0x7ff0fc060360 2024-02-20T12:59:27.676 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:27.633+0000 7ff0fb7fe700 1 --2- 172.21.15.71:0/2336784255 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff0fc060360 0x7ff0fc0f6a30 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 rx=0x7ff0ec009610 tx=0x7ff0ec004d10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:59:27.676 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:27.633+0000 7ff0f3fff700 1 -- 172.21.15.71:0/2336784255 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff0ec018070 con 0x7ff0fc060360 2024-02-20T12:59:27.676 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:27.633+0000 7ff100812700 1 -- 172.21.15.71:0/2336784255 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0fc0f5050 con 0x7ff0fc060360 2024-02-20T12:59:27.676 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:27.633+0000 7ff100812700 1 -- 172.21.15.71:0/2336784255 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0fc0f54e0 con 0x7ff0fc060360 2024-02-20T12:59:27.676 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:27.633+0000 7ff0f3fff700 1 -- 172.21.15.71:0/2336784255 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(25 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7ff0ec0048f0 con 0x7ff0fc060360 2024-02-20T12:59:27.676 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:27.633+0000 7ff0f3fff700 1 -- 172.21.15.71:0/2336784255 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff0ec01d420 con 0x7ff0fc060360 2024-02-20T12:59:27.676 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:27.634+0000 7ff0f3fff700 1 -- 172.21.15.71:0/2336784255 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 37151+0+0 (secure 0 0 0) 0x7ff0ec024060 con 0x7ff0fc060360 2024-02-20T12:59:27.676 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:27.634+0000 7ff100812700 1 -- 172.21.15.71:0/2336784255 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0fc0591e0 con 0x7ff0fc060360 2024-02-20T12:59:27.676 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:27.634+0000 7ff0f3fff700 1 --2- 172.21.15.71:0/2336784255 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7ff0e8031090 0x7ff0e8033550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:59:27.676 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:27.634+0000 7ff0f3fff700 1 -- 172.21.15.71:0/2336784255 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1065+0+0 (secure 0 0 0) 0x7ff0ec0426b0 con 0x7ff0fc060360 2024-02-20T12:59:27.676 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:27.634+0000 7ff0faffd700 1 --2- 172.21.15.71:0/2336784255 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7ff0e8031090 0x7ff0e8033550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:59:27.677 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:27.635+0000 7ff0faffd700 1 --2- 172.21.15.71:0/2336784255 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7ff0e8031090 0x7ff0e8033550 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 rx=0x7ff0e40096a0 tx=0x7ff0e4006b40).ready entity=mgr.14156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:59:27.677 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:27.639+0000 7ff0f3fff700 1 -- 172.21.15.71:0/2336784255 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+153237 (secure 0 0 0) 0x7ff0ec011320 con 0x7ff0fc060360 2024-02-20T12:59:27.849 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:27.848+0000 7ff100812700 1 -- 172.21.15.71:0/2336784255 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7ff0fc0fbeb0 con 0x7ff0fc060360 2024-02-20T12:59:27.850 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:27.849+0000 7ff0f3fff700 1 -- 172.21.15.71:0/2336784255 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7ff0ec022020 con 0x7ff0fc060360 2024-02-20T12:59:27.850 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-20T12:59:27.850 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":1,"fsid":"680ee126-cfee-11ee-95bb-87774f69a715","modified":"2024-02-20T12:49:44.699059Z","created":"2024-02-20T12:49:44.699059Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi064","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:3300","nonce":0},{"type":"v1","addr":"172.21.15.64:6789","nonce":0}]},"addr":"172.21.15.64:6789/0","public_addr":"172.21.15.64:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-20T12:59:27.853 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:27.850+0000 7ff100812700 1 -- 172.21.15.71:0/2336784255 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7ff0e8031090 msgr2=0x7ff0e8033550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:59:27.853 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:27.850+0000 7ff100812700 1 --2- 172.21.15.71:0/2336784255 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7ff0e8031090 0x7ff0e8033550 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 rx=0x7ff0e40096a0 tx=0x7ff0e4006b40).stop 2024-02-20T12:59:27.853 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:27.850+0000 7ff100812700 1 -- 172.21.15.71:0/2336784255 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff0fc060360 msgr2=0x7ff0fc0f6a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:59:27.853 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:27.850+0000 7ff100812700 1 --2- 172.21.15.71:0/2336784255 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff0fc060360 0x7ff0fc0f6a30 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 rx=0x7ff0ec009610 tx=0x7ff0ec004d10).stop 2024-02-20T12:59:27.853 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:27.850+0000 7ff100812700 1 -- 172.21.15.71:0/2336784255 shutdown_connections 2024-02-20T12:59:27.853 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:27.850+0000 7ff100812700 1 --2- 172.21.15.71:0/2336784255 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7ff0e8031090 0x7ff0e8033550 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:59:27.853 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:27.850+0000 7ff100812700 1 --2- 172.21.15.71:0/2336784255 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff0fc060360 0x7ff0fc0f6a30 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:59:27.853 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:27.850+0000 7ff100812700 1 -- 172.21.15.71:0/2336784255 >> 172.21.15.71:0/2336784255 conn(0x7ff0fc0f0a30 msgr2=0x7ff0fc0f14e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:59:27.853 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:27.851+0000 7ff100812700 1 -- 172.21.15.71:0/2336784255 shutdown_connections 2024-02-20T12:59:27.853 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:27.851+0000 7ff100812700 1 -- 172.21.15.71:0/2336784255 wait complete. 2024-02-20T12:59:27.853 INFO:teuthology.orchestra.run.smithi071.stderr:dumped monmap epoch 1 2024-02-20T12:59:28.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:59:28 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:59:27.850405+0000 mon.smithi064 (mon.0) 260 : audit [DBG] from='client.? 172.21.15.71:0/2336784255' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-20T12:59:29.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:59:29 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:59:27.787668+0000 mgr.smithi064.vriyor (mgr.14156) 268 : cluster [DBG] pgmap v251: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:59:29.652 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-20T12:59:29.653 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/daemon-base:latest-pacific shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 680ee126-cfee-11ee-95bb-87774f69a715 -- ceph mon dump -f json 2024-02-20T12:59:31.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:59:31 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:59:29.788114+0000 mgr.smithi064.vriyor (mgr.14156) 269 : cluster [DBG] pgmap v252: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:59:31.676 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:31.674+0000 7ff455ab9700 1 -- 172.21.15.71:0/3038850926 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff4500f5480 msgr2=0x7ff4500f58a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:59:31.677 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:31.674+0000 7ff455ab9700 1 --2- 172.21.15.71:0/3038850926 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff4500f5480 0x7ff4500f58a0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 rx=0x7ff440009960 tx=0x7ff440004d10).stop 2024-02-20T12:59:31.677 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:31.674+0000 7ff455ab9700 1 -- 172.21.15.71:0/3038850926 shutdown_connections 2024-02-20T12:59:31.677 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:31.674+0000 7ff455ab9700 1 --2- 172.21.15.71:0/3038850926 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff4500f5480 0x7ff4500f58a0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:59:31.677 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:31.674+0000 7ff455ab9700 1 -- 172.21.15.71:0/3038850926 >> 172.21.15.71:0/3038850926 conn(0x7ff4500f0be0 msgr2=0x7ff4500f3000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:59:31.677 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:31.675+0000 7ff455ab9700 1 -- 172.21.15.71:0/3038850926 shutdown_connections 2024-02-20T12:59:31.677 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:31.675+0000 7ff455ab9700 1 -- 172.21.15.71:0/3038850926 wait complete. 2024-02-20T12:59:31.677 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:31.675+0000 7ff455ab9700 1 Processor -- start 2024-02-20T12:59:31.677 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:31.675+0000 7ff455ab9700 1 -- start start 2024-02-20T12:59:31.677 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:31.675+0000 7ff455ab9700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff4500f5480 0x7ff4501094f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:59:31.677 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:31.675+0000 7ff455ab9700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff450060320 con 0x7ff4500f5480 2024-02-20T12:59:31.679 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:31.676+0000 7ff44f7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff4500f5480 0x7ff4501094f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:59:31.679 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:31.676+0000 7ff44f7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff4500f5480 0x7ff4501094f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.71:56272/0 (socket says 172.21.15.71:56272) 2024-02-20T12:59:31.679 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:31.676+0000 7ff44f7fe700 1 -- 172.21.15.71:0/1618489325 learned_addr learned my addr 172.21.15.71:0/1618489325 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-02-20T12:59:31.679 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:31.676+0000 7ff44f7fe700 1 -- 172.21.15.71:0/1618489325 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff440009640 con 0x7ff4500f5480 2024-02-20T12:59:31.679 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:31.676+0000 7ff44f7fe700 1 --2- 172.21.15.71:0/1618489325 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff4500f5480 0x7ff4501094f0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 rx=0x7ff440009610 tx=0x7ff440004d10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:59:31.679 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:31.676+0000 7ff447fff700 1 -- 172.21.15.71:0/1618489325 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff440018070 con 0x7ff4500f5480 2024-02-20T12:59:31.679 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:31.676+0000 7ff455ab9700 1 -- 172.21.15.71:0/1618489325 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff450060520 con 0x7ff4500f5480 2024-02-20T12:59:31.680 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:31.676+0000 7ff447fff700 1 -- 172.21.15.71:0/1618489325 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(25 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7ff44000b7e0 con 0x7ff4500f5480 2024-02-20T12:59:31.680 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:31.676+0000 7ff447fff700 1 -- 172.21.15.71:0/1618489325 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff44001d420 con 0x7ff4500f5480 2024-02-20T12:59:31.680 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:31.676+0000 7ff455ab9700 1 -- 172.21.15.71:0/1618489325 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff450109ba0 con 0x7ff4500f5480 2024-02-20T12:59:31.680 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:31.677+0000 7ff447fff700 1 -- 172.21.15.71:0/1618489325 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 37151+0+0 (secure 0 0 0) 0x7ff440024060 con 0x7ff4500f5480 2024-02-20T12:59:31.680 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:31.677+0000 7ff455ab9700 1 -- 172.21.15.71:0/1618489325 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff450045bb0 con 0x7ff4500f5480 2024-02-20T12:59:31.680 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:31.677+0000 7ff447fff700 1 --2- 172.21.15.71:0/1618489325 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7ff43c0310e0 0x7ff43c0335a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:59:31.680 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:31.677+0000 7ff447fff700 1 -- 172.21.15.71:0/1618489325 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1065+0+0 (secure 0 0 0) 0x7ff440042630 con 0x7ff4500f5480 2024-02-20T12:59:31.680 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:31.678+0000 7ff44effd700 1 --2- 172.21.15.71:0/1618489325 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7ff43c0310e0 0x7ff43c0335a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:59:31.680 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:31.678+0000 7ff44effd700 1 --2- 172.21.15.71:0/1618489325 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7ff43c0310e0 0x7ff43c0335a0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 rx=0x7ff4380096a0 tx=0x7ff438006b40).ready entity=mgr.14156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:59:31.684 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:31.682+0000 7ff447fff700 1 -- 172.21.15.71:0/1618489325 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+153237 (secure 0 0 0) 0x7ff4400048d0 con 0x7ff4500f5480 2024-02-20T12:59:31.893 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:31.891+0000 7ff455ab9700 1 -- 172.21.15.71:0/1618489325 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7ff4500faf70 con 0x7ff4500f5480 2024-02-20T12:59:31.893 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:31.892+0000 7ff447fff700 1 -- 172.21.15.71:0/1618489325 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7ff4400048d0 con 0x7ff4500f5480 2024-02-20T12:59:31.894 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-20T12:59:31.894 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":1,"fsid":"680ee126-cfee-11ee-95bb-87774f69a715","modified":"2024-02-20T12:49:44.699059Z","created":"2024-02-20T12:49:44.699059Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi064","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:3300","nonce":0},{"type":"v1","addr":"172.21.15.64:6789","nonce":0}]},"addr":"172.21.15.64:6789/0","public_addr":"172.21.15.64:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-20T12:59:31.895 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:31.894+0000 7ff455ab9700 1 -- 172.21.15.71:0/1618489325 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7ff43c0310e0 msgr2=0x7ff43c0335a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:59:31.895 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:31.894+0000 7ff455ab9700 1 --2- 172.21.15.71:0/1618489325 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7ff43c0310e0 0x7ff43c0335a0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 rx=0x7ff4380096a0 tx=0x7ff438006b40).stop 2024-02-20T12:59:31.896 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:31.894+0000 7ff455ab9700 1 -- 172.21.15.71:0/1618489325 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff4500f5480 msgr2=0x7ff4501094f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:59:31.896 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:31.894+0000 7ff455ab9700 1 --2- 172.21.15.71:0/1618489325 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff4500f5480 0x7ff4501094f0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 rx=0x7ff440009610 tx=0x7ff440004d10).stop 2024-02-20T12:59:31.896 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:31.894+0000 7ff455ab9700 1 -- 172.21.15.71:0/1618489325 shutdown_connections 2024-02-20T12:59:31.896 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:31.894+0000 7ff455ab9700 1 --2- 172.21.15.71:0/1618489325 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7ff43c0310e0 0x7ff43c0335a0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:59:31.896 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:31.894+0000 7ff455ab9700 1 --2- 172.21.15.71:0/1618489325 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff4500f5480 0x7ff4501094f0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:59:31.896 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:31.894+0000 7ff455ab9700 1 -- 172.21.15.71:0/1618489325 >> 172.21.15.71:0/1618489325 conn(0x7ff4500f0be0 msgr2=0x7ff4500f16d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:59:31.896 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:31.894+0000 7ff455ab9700 1 -- 172.21.15.71:0/1618489325 shutdown_connections 2024-02-20T12:59:31.896 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:31.894+0000 7ff455ab9700 1 -- 172.21.15.71:0/1618489325 wait complete. 2024-02-20T12:59:31.896 INFO:teuthology.orchestra.run.smithi071.stderr:dumped monmap epoch 1 2024-02-20T12:59:32.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:59:32 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:59:31.893911+0000 mon.smithi064 (mon.0) 261 : audit [DBG] from='client.? 172.21.15.71:0/1618489325' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-20T12:59:33.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:59:33 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:59:31.788554+0000 mgr.smithi064.vriyor (mgr.14156) 270 : cluster [DBG] pgmap v253: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:59:33.668 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-20T12:59:33.668 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/daemon-base:latest-pacific shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 680ee126-cfee-11ee-95bb-87774f69a715 -- ceph mon dump -f json 2024-02-20T12:59:35.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:59:35 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:59:33.788988+0000 mgr.smithi064.vriyor (mgr.14156) 271 : cluster [DBG] pgmap v254: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:59:35.676 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:35.664+0000 7f3e7759e700 1 -- 172.21.15.71:0/1613347405 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3e780f52d0 msgr2=0x7f3e780f56f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:59:35.676 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:35.664+0000 7f3e7759e700 1 --2- 172.21.15.71:0/1613347405 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3e780f52d0 0x7f3e780f56f0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 rx=0x7f3e6c009fb0 tx=0x7f3e6c009b30).stop 2024-02-20T12:59:35.676 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:35.664+0000 7f3e7759e700 1 -- 172.21.15.71:0/1613347405 shutdown_connections 2024-02-20T12:59:35.677 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:35.664+0000 7f3e7759e700 1 --2- 172.21.15.71:0/1613347405 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3e780f52d0 0x7f3e780f56f0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:59:35.677 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:35.664+0000 7f3e7759e700 1 -- 172.21.15.71:0/1613347405 >> 172.21.15.71:0/1613347405 conn(0x7f3e780f0a30 msgr2=0x7f3e780f2e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:59:35.677 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:35.665+0000 7f3e7759e700 1 -- 172.21.15.71:0/1613347405 shutdown_connections 2024-02-20T12:59:35.677 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:35.665+0000 7f3e7759e700 1 -- 172.21.15.71:0/1613347405 wait complete. 2024-02-20T12:59:35.677 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:35.665+0000 7f3e7759e700 1 Processor -- start 2024-02-20T12:59:35.677 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:35.665+0000 7f3e7759e700 1 -- start start 2024-02-20T12:59:35.677 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:35.665+0000 7f3e7759e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3e780f52d0 0x7f3e78106600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:59:35.677 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:35.665+0000 7f3e7759e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e78106af0 con 0x7f3e780f52d0 2024-02-20T12:59:35.677 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:35.665+0000 7f3e7659c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3e780f52d0 0x7f3e78106600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:59:35.677 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:35.665+0000 7f3e7659c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3e780f52d0 0x7f3e78106600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.71:56300/0 (socket says 172.21.15.71:56300) 2024-02-20T12:59:35.677 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:35.665+0000 7f3e7659c700 1 -- 172.21.15.71:0/1332019159 learned_addr learned my addr 172.21.15.71:0/1332019159 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-02-20T12:59:35.677 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:35.666+0000 7f3e7659c700 1 -- 172.21.15.71:0/1332019159 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e6c005040 con 0x7f3e780f52d0 2024-02-20T12:59:35.677 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:35.666+0000 7f3e7659c700 1 --2- 172.21.15.71:0/1332019159 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3e780f52d0 0x7f3e78106600 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 rx=0x7f3e6c0089f0 tx=0x7f3e6c00b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:59:35.677 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:35.666+0000 7f3e66ffd700 1 -- 172.21.15.71:0/1332019159 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3e6c009320 con 0x7f3e780f52d0 2024-02-20T12:59:35.677 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:35.666+0000 7f3e7759e700 1 -- 172.21.15.71:0/1332019159 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e78106cf0 con 0x7f3e780f52d0 2024-02-20T12:59:35.678 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:35.666+0000 7f3e66ffd700 1 -- 172.21.15.71:0/1332019159 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(25 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f3e6c013ec0 con 0x7f3e780f52d0 2024-02-20T12:59:35.678 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:35.666+0000 7f3e66ffd700 1 -- 172.21.15.71:0/1332019159 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3e6c014d30 con 0x7f3e780f52d0 2024-02-20T12:59:35.678 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:35.666+0000 7f3e7759e700 1 -- 172.21.15.71:0/1332019159 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e78107180 con 0x7f3e780f52d0 2024-02-20T12:59:35.678 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:35.667+0000 7f3e66ffd700 1 -- 172.21.15.71:0/1332019159 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 37151+0+0 (secure 0 0 0) 0x7f3e6c0095f0 con 0x7f3e780f52d0 2024-02-20T12:59:35.678 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:35.667+0000 7f3e7759e700 1 -- 172.21.15.71:0/1332019159 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e78045bb0 con 0x7f3e780f52d0 2024-02-20T12:59:35.678 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:35.667+0000 7f3e66ffd700 1 --2- 172.21.15.71:0/1332019159 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f3e60031090 0x7f3e60033550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:59:35.678 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:35.667+0000 7f3e66ffd700 1 -- 172.21.15.71:0/1332019159 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1065+0+0 (secure 0 0 0) 0x7f3e6c0423e0 con 0x7f3e780f52d0 2024-02-20T12:59:35.678 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:35.667+0000 7f3e75d9b700 1 --2- 172.21.15.71:0/1332019159 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f3e60031090 0x7f3e60033550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:59:35.678 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:35.668+0000 7f3e75d9b700 1 --2- 172.21.15.71:0/1332019159 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f3e60031090 0x7f3e60033550 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 rx=0x7f3e680096a0 tx=0x7f3e68006b40).ready entity=mgr.14156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:59:35.678 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:35.672+0000 7f3e66ffd700 1 -- 172.21.15.71:0/1332019159 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+153237 (secure 0 0 0) 0x7f3e6c0268c0 con 0x7f3e780f52d0 2024-02-20T12:59:35.880 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:35.879+0000 7f3e7759e700 1 -- 172.21.15.71:0/1332019159 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f3e78108030 con 0x7f3e780f52d0 2024-02-20T12:59:35.881 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:35.880+0000 7f3e66ffd700 1 -- 172.21.15.71:0/1332019159 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7f3e6c01d380 con 0x7f3e780f52d0 2024-02-20T12:59:35.881 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-20T12:59:35.881 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":1,"fsid":"680ee126-cfee-11ee-95bb-87774f69a715","modified":"2024-02-20T12:49:44.699059Z","created":"2024-02-20T12:49:44.699059Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi064","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:3300","nonce":0},{"type":"v1","addr":"172.21.15.64:6789","nonce":0}]},"addr":"172.21.15.64:6789/0","public_addr":"172.21.15.64:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-20T12:59:35.883 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:35.881+0000 7f3e7759e700 1 -- 172.21.15.71:0/1332019159 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f3e60031090 msgr2=0x7f3e60033550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:59:35.883 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:35.881+0000 7f3e7759e700 1 --2- 172.21.15.71:0/1332019159 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f3e60031090 0x7f3e60033550 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 rx=0x7f3e680096a0 tx=0x7f3e68006b40).stop 2024-02-20T12:59:35.883 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:35.881+0000 7f3e7759e700 1 -- 172.21.15.71:0/1332019159 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3e780f52d0 msgr2=0x7f3e78106600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:59:35.883 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:35.881+0000 7f3e7759e700 1 --2- 172.21.15.71:0/1332019159 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3e780f52d0 0x7f3e78106600 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 rx=0x7f3e6c0089f0 tx=0x7f3e6c00b040).stop 2024-02-20T12:59:35.883 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:35.882+0000 7f3e7759e700 1 -- 172.21.15.71:0/1332019159 shutdown_connections 2024-02-20T12:59:35.883 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:35.882+0000 7f3e7759e700 1 --2- 172.21.15.71:0/1332019159 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f3e60031090 0x7f3e60033550 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:59:35.883 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:35.882+0000 7f3e7759e700 1 --2- 172.21.15.71:0/1332019159 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3e780f52d0 0x7f3e78106600 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:59:35.883 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:35.882+0000 7f3e7759e700 1 -- 172.21.15.71:0/1332019159 >> 172.21.15.71:0/1332019159 conn(0x7f3e780f0a30 msgr2=0x7f3e7805fec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:59:35.883 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:35.882+0000 7f3e7759e700 1 -- 172.21.15.71:0/1332019159 shutdown_connections 2024-02-20T12:59:35.883 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:35.882+0000 7f3e7759e700 1 -- 172.21.15.71:0/1332019159 wait complete. 2024-02-20T12:59:35.884 INFO:teuthology.orchestra.run.smithi071.stderr:dumped monmap epoch 1 2024-02-20T12:59:36.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:59:36 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:59:35.881333+0000 mon.smithi064 (mon.0) 262 : audit [DBG] from='client.? 172.21.15.71:0/1332019159' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-20T12:59:37.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:59:37 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:59:35.789376+0000 mgr.smithi064.vriyor (mgr.14156) 272 : cluster [DBG] pgmap v255: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:59:37.610 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-20T12:59:37.610 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/daemon-base:latest-pacific shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 680ee126-cfee-11ee-95bb-87774f69a715 -- ceph mon dump -f json 2024-02-20T12:59:39.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:59:39 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:59:37.789704+0000 mgr.smithi064.vriyor (mgr.14156) 273 : cluster [DBG] pgmap v256: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:59:39.592 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:39.591+0000 7fea6e3e0700 1 -- 172.21.15.71:0/2129810381 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fea680f6590 msgr2=0x7fea680f69b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:59:39.592 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:39.591+0000 7fea6e3e0700 1 --2- 172.21.15.71:0/2129810381 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fea680f6590 0x7fea680f69b0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 rx=0x7fea58004760 tx=0x7fea58009ad0).stop 2024-02-20T12:59:39.593 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:39.591+0000 7fea6e3e0700 1 -- 172.21.15.71:0/2129810381 shutdown_connections 2024-02-20T12:59:39.593 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:39.591+0000 7fea6e3e0700 1 --2- 172.21.15.71:0/2129810381 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fea680f6590 0x7fea680f69b0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:59:39.593 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:39.591+0000 7fea6e3e0700 1 -- 172.21.15.71:0/2129810381 >> 172.21.15.71:0/2129810381 conn(0x7fea680f1cd0 msgr2=0x7fea680f4110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:59:39.593 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:39.591+0000 7fea6e3e0700 1 -- 172.21.15.71:0/2129810381 shutdown_connections 2024-02-20T12:59:39.593 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:39.591+0000 7fea6e3e0700 1 -- 172.21.15.71:0/2129810381 wait complete. 2024-02-20T12:59:39.593 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:39.591+0000 7fea6e3e0700 1 Processor -- start 2024-02-20T12:59:39.595 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:39.591+0000 7fea6e3e0700 1 -- start start 2024-02-20T12:59:39.595 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:39.591+0000 7fea6e3e0700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fea680f6590 0x7fea681067c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:59:39.595 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:39.591+0000 7fea6e3e0700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea68106cb0 con 0x7fea680f6590 2024-02-20T12:59:39.595 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:39.592+0000 7fea67fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fea680f6590 0x7fea681067c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:59:39.595 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:39.592+0000 7fea67fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fea680f6590 0x7fea681067c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.71:42686/0 (socket says 172.21.15.71:42686) 2024-02-20T12:59:39.596 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:39.592+0000 7fea67fff700 1 -- 172.21.15.71:0/3947446310 learned_addr learned my addr 172.21.15.71:0/3947446310 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-02-20T12:59:39.596 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:39.592+0000 7fea67fff700 1 -- 172.21.15.71:0/3947446310 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea58009d70 con 0x7fea680f6590 2024-02-20T12:59:39.596 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:39.592+0000 7fea67fff700 1 --2- 172.21.15.71:0/3947446310 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fea680f6590 0x7fea681067c0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 rx=0x7fea58004170 tx=0x7fea58009ad0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:59:39.596 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:39.592+0000 7fea64ff9700 1 -- 172.21.15.71:0/3947446310 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fea5800e040 con 0x7fea680f6590 2024-02-20T12:59:39.596 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:39.592+0000 7fea6e3e0700 1 -- 172.21.15.71:0/3947446310 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea68106eb0 con 0x7fea680f6590 2024-02-20T12:59:39.596 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:39.592+0000 7fea64ff9700 1 -- 172.21.15.71:0/3947446310 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(25 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7fea58008ef0 con 0x7fea680f6590 2024-02-20T12:59:39.596 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:39.592+0000 7fea64ff9700 1 -- 172.21.15.71:0/3947446310 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fea58014c90 con 0x7fea680f6590 2024-02-20T12:59:39.596 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:39.592+0000 7fea6e3e0700 1 -- 172.21.15.71:0/3947446310 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea68107340 con 0x7fea680f6590 2024-02-20T12:59:39.596 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:39.593+0000 7fea64ff9700 1 -- 172.21.15.71:0/3947446310 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 37151+0+0 (secure 0 0 0) 0x7fea58014420 con 0x7fea680f6590 2024-02-20T12:59:39.596 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:39.593+0000 7fea6e3e0700 1 -- 172.21.15.71:0/3947446310 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea68045bb0 con 0x7fea680f6590 2024-02-20T12:59:39.596 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:39.593+0000 7fea64ff9700 1 --2- 172.21.15.71:0/3947446310 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fea50031090 0x7fea50033550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:59:39.596 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:39.593+0000 7fea64ff9700 1 -- 172.21.15.71:0/3947446310 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1065+0+0 (secure 0 0 0) 0x7fea58042320 con 0x7fea680f6590 2024-02-20T12:59:39.596 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:39.594+0000 7fea677fe700 1 --2- 172.21.15.71:0/3947446310 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fea50031090 0x7fea50033550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:59:39.596 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:39.594+0000 7fea677fe700 1 --2- 172.21.15.71:0/3947446310 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fea50031090 0x7fea50033550 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 rx=0x7fea5c0096a0 tx=0x7fea5c006b40).ready entity=mgr.14156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:59:39.599 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:39.599+0000 7fea64ff9700 1 -- 172.21.15.71:0/3947446310 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+153237 (secure 0 0 0) 0x7fea58026300 con 0x7fea680f6590 2024-02-20T12:59:39.804 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:39.802+0000 7fea6e3e0700 1 -- 172.21.15.71:0/3947446310 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fea68108190 con 0x7fea680f6590 2024-02-20T12:59:39.805 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:39.804+0000 7fea64ff9700 1 -- 172.21.15.71:0/3947446310 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7fea58012b20 con 0x7fea680f6590 2024-02-20T12:59:39.806 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-20T12:59:39.806 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":1,"fsid":"680ee126-cfee-11ee-95bb-87774f69a715","modified":"2024-02-20T12:49:44.699059Z","created":"2024-02-20T12:49:44.699059Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi064","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:3300","nonce":0},{"type":"v1","addr":"172.21.15.64:6789","nonce":0}]},"addr":"172.21.15.64:6789/0","public_addr":"172.21.15.64:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-20T12:59:39.807 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:39.804+0000 7fea6e3e0700 1 -- 172.21.15.71:0/3947446310 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fea50031090 msgr2=0x7fea50033550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:59:39.807 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:39.804+0000 7fea6e3e0700 1 --2- 172.21.15.71:0/3947446310 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fea50031090 0x7fea50033550 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 rx=0x7fea5c0096a0 tx=0x7fea5c006b40).stop 2024-02-20T12:59:39.807 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:39.804+0000 7fea6e3e0700 1 -- 172.21.15.71:0/3947446310 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fea680f6590 msgr2=0x7fea681067c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:59:39.808 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:39.804+0000 7fea6e3e0700 1 --2- 172.21.15.71:0/3947446310 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fea680f6590 0x7fea681067c0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 rx=0x7fea58004170 tx=0x7fea58009ad0).stop 2024-02-20T12:59:39.808 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:39.804+0000 7fea6e3e0700 1 -- 172.21.15.71:0/3947446310 shutdown_connections 2024-02-20T12:59:39.808 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:39.804+0000 7fea6e3e0700 1 --2- 172.21.15.71:0/3947446310 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fea50031090 0x7fea50033550 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:59:39.808 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:39.804+0000 7fea6e3e0700 1 --2- 172.21.15.71:0/3947446310 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fea680f6590 0x7fea681067c0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:59:39.808 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:39.804+0000 7fea6e3e0700 1 -- 172.21.15.71:0/3947446310 >> 172.21.15.71:0/3947446310 conn(0x7fea680f1cd0 msgr2=0x7fea680f27c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:59:39.809 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:39.804+0000 7fea6e3e0700 1 -- 172.21.15.71:0/3947446310 shutdown_connections 2024-02-20T12:59:39.809 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:39.804+0000 7fea6e3e0700 1 -- 172.21.15.71:0/3947446310 wait complete. 2024-02-20T12:59:39.809 INFO:teuthology.orchestra.run.smithi071.stderr:dumped monmap epoch 1 2024-02-20T12:59:40.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:59:40 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:59:39.804606+0000 mon.smithi064 (mon.0) 263 : audit [DBG] from='client.? 172.21.15.71:0/3947446310' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-20T12:59:41.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:59:41 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:59:39.790129+0000 mgr.smithi064.vriyor (mgr.14156) 274 : cluster [DBG] pgmap v257: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:59:41.484 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-20T12:59:41.484 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/daemon-base:latest-pacific shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 680ee126-cfee-11ee-95bb-87774f69a715 -- ceph mon dump -f json 2024-02-20T12:59:43.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:59:43 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:59:41.790449+0000 mgr.smithi064.vriyor (mgr.14156) 275 : cluster [DBG] pgmap v258: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:59:43.434 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:43.384+0000 7fdf3d03c700 1 -- 172.21.15.71:0/3891357163 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdf380f52d0 msgr2=0x7fdf380f56f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:59:43.434 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:43.384+0000 7fdf3d03c700 1 --2- 172.21.15.71:0/3891357163 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdf380f52d0 0x7fdf380f56f0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 rx=0x7fdf28009fb0 tx=0x7fdf28009b30).stop 2024-02-20T12:59:43.434 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:43.384+0000 7fdf3d03c700 1 -- 172.21.15.71:0/3891357163 shutdown_connections 2024-02-20T12:59:43.434 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:43.384+0000 7fdf3d03c700 1 --2- 172.21.15.71:0/3891357163 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdf380f52d0 0x7fdf380f56f0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:59:43.434 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:43.384+0000 7fdf3d03c700 1 -- 172.21.15.71:0/3891357163 >> 172.21.15.71:0/3891357163 conn(0x7fdf380f0a30 msgr2=0x7fdf380f2e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:59:43.435 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:43.384+0000 7fdf3d03c700 1 -- 172.21.15.71:0/3891357163 shutdown_connections 2024-02-20T12:59:43.435 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:43.384+0000 7fdf3d03c700 1 -- 172.21.15.71:0/3891357163 wait complete. 2024-02-20T12:59:43.435 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:43.384+0000 7fdf3d03c700 1 Processor -- start 2024-02-20T12:59:43.435 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:43.384+0000 7fdf3d03c700 1 -- start start 2024-02-20T12:59:43.435 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:43.385+0000 7fdf3d03c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdf380f52d0 0x7fdf38105510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:59:43.435 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:43.385+0000 7fdf3d03c700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf38105a00 con 0x7fdf380f52d0 2024-02-20T12:59:43.435 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:43.385+0000 7fdf37fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdf380f52d0 0x7fdf38105510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:59:43.435 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:43.385+0000 7fdf37fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdf380f52d0 0x7fdf38105510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.71:42692/0 (socket says 172.21.15.71:42692) 2024-02-20T12:59:43.435 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:43.385+0000 7fdf37fff700 1 -- 172.21.15.71:0/1937339809 learned_addr learned my addr 172.21.15.71:0/1937339809 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-02-20T12:59:43.435 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:43.385+0000 7fdf37fff700 1 -- 172.21.15.71:0/1937339809 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf28005040 con 0x7fdf380f52d0 2024-02-20T12:59:43.435 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:43.385+0000 7fdf37fff700 1 --2- 172.21.15.71:0/1937339809 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdf380f52d0 0x7fdf38105510 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 rx=0x7fdf280089f0 tx=0x7fdf2800b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:59:43.435 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:43.385+0000 7fdf34ff9700 1 -- 172.21.15.71:0/1937339809 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fdf28009320 con 0x7fdf380f52d0 2024-02-20T12:59:43.435 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:43.385+0000 7fdf3d03c700 1 -- 172.21.15.71:0/1937339809 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf38105c00 con 0x7fdf380f52d0 2024-02-20T12:59:43.435 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:43.386+0000 7fdf34ff9700 1 -- 172.21.15.71:0/1937339809 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(25 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7fdf28013ec0 con 0x7fdf380f52d0 2024-02-20T12:59:43.436 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:43.386+0000 7fdf34ff9700 1 -- 172.21.15.71:0/1937339809 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fdf28014d30 con 0x7fdf380f52d0 2024-02-20T12:59:43.436 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:43.386+0000 7fdf3d03c700 1 -- 172.21.15.71:0/1937339809 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf38106090 con 0x7fdf380f52d0 2024-02-20T12:59:43.436 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:43.386+0000 7fdf34ff9700 1 -- 172.21.15.71:0/1937339809 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 37151+0+0 (secure 0 0 0) 0x7fdf280263a0 con 0x7fdf380f52d0 2024-02-20T12:59:43.436 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:43.386+0000 7fdf34ff9700 1 --2- 172.21.15.71:0/1937339809 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fdf20031090 0x7fdf20033550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:59:43.436 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:43.387+0000 7fdf34ff9700 1 -- 172.21.15.71:0/1937339809 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1065+0+0 (secure 0 0 0) 0x7fdf280423a0 con 0x7fdf380f52d0 2024-02-20T12:59:43.436 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:43.387+0000 7fdf3d03c700 1 -- 172.21.15.71:0/1937339809 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf380591e0 con 0x7fdf380f52d0 2024-02-20T12:59:43.436 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:43.387+0000 7fdf377fe700 1 --2- 172.21.15.71:0/1937339809 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fdf20031090 0x7fdf20033550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:59:43.436 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:43.387+0000 7fdf377fe700 1 --2- 172.21.15.71:0/1937339809 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fdf20031090 0x7fdf20033550 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 rx=0x7fdf2c0096a0 tx=0x7fdf2c006b40).ready entity=mgr.14156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:59:43.436 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:43.391+0000 7fdf34ff9700 1 -- 172.21.15.71:0/1937339809 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+153237 (secure 0 0 0) 0x7fdf280266d0 con 0x7fdf380f52d0 2024-02-20T12:59:43.599 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:43.599+0000 7fdf3d03c700 1 -- 172.21.15.71:0/1937339809 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fdf38045bb0 con 0x7fdf380f52d0 2024-02-20T12:59:43.600 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:43.599+0000 7fdf34ff9700 1 -- 172.21.15.71:0/1937339809 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7fdf2802b020 con 0x7fdf380f52d0 2024-02-20T12:59:43.600 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-20T12:59:43.600 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":1,"fsid":"680ee126-cfee-11ee-95bb-87774f69a715","modified":"2024-02-20T12:49:44.699059Z","created":"2024-02-20T12:49:44.699059Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi064","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:3300","nonce":0},{"type":"v1","addr":"172.21.15.64:6789","nonce":0}]},"addr":"172.21.15.64:6789/0","public_addr":"172.21.15.64:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-20T12:59:43.601 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:43.601+0000 7fdf3d03c700 1 -- 172.21.15.71:0/1937339809 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fdf20031090 msgr2=0x7fdf20033550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:59:43.601 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:43.601+0000 7fdf3d03c700 1 --2- 172.21.15.71:0/1937339809 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fdf20031090 0x7fdf20033550 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 rx=0x7fdf2c0096a0 tx=0x7fdf2c006b40).stop 2024-02-20T12:59:43.601 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:43.601+0000 7fdf3d03c700 1 -- 172.21.15.71:0/1937339809 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdf380f52d0 msgr2=0x7fdf38105510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:59:43.602 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:43.601+0000 7fdf3d03c700 1 --2- 172.21.15.71:0/1937339809 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdf380f52d0 0x7fdf38105510 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 rx=0x7fdf280089f0 tx=0x7fdf2800b040).stop 2024-02-20T12:59:43.602 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:43.601+0000 7fdf3d03c700 1 -- 172.21.15.71:0/1937339809 shutdown_connections 2024-02-20T12:59:43.602 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:43.601+0000 7fdf3d03c700 1 --2- 172.21.15.71:0/1937339809 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fdf20031090 0x7fdf20033550 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:59:43.602 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:43.601+0000 7fdf3d03c700 1 --2- 172.21.15.71:0/1937339809 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdf380f52d0 0x7fdf38105510 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:59:43.602 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:43.601+0000 7fdf3d03c700 1 -- 172.21.15.71:0/1937339809 >> 172.21.15.71:0/1937339809 conn(0x7fdf380f0a30 msgr2=0x7fdf380f1550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:59:43.602 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:43.601+0000 7fdf3d03c700 1 -- 172.21.15.71:0/1937339809 shutdown_connections 2024-02-20T12:59:43.602 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:43.601+0000 7fdf3d03c700 1 -- 172.21.15.71:0/1937339809 wait complete. 2024-02-20T12:59:43.602 INFO:teuthology.orchestra.run.smithi071.stderr:dumped monmap epoch 1 2024-02-20T12:59:44.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:59:44 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:59:43.600330+0000 mon.smithi064 (mon.0) 264 : audit [DBG] from='client.? 172.21.15.71:0/1937339809' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-20T12:59:45.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:59:45 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:59:43.790707+0000 mgr.smithi064.vriyor (mgr.14156) 276 : cluster [DBG] pgmap v259: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:59:45.389 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-20T12:59:45.390 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/daemon-base:latest-pacific shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 680ee126-cfee-11ee-95bb-87774f69a715 -- ceph mon dump -f json 2024-02-20T12:59:47.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:59:47 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:59:45.791089+0000 mgr.smithi064.vriyor (mgr.14156) 277 : cluster [DBG] pgmap v260: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:59:47.432 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:47.430+0000 7fccd05fe700 1 -- 172.21.15.71:0/4042958904 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fccc80f6590 msgr2=0x7fccc80f69b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:59:47.432 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:47.430+0000 7fccd05fe700 1 --2- 172.21.15.71:0/4042958904 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fccc80f6590 0x7fccc80f69b0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 rx=0x7fccb8009fb0 tx=0x7fccb8009b30).stop 2024-02-20T12:59:47.433 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:47.430+0000 7fccd05fe700 1 -- 172.21.15.71:0/4042958904 shutdown_connections 2024-02-20T12:59:47.433 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:47.430+0000 7fccd05fe700 1 --2- 172.21.15.71:0/4042958904 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fccc80f6590 0x7fccc80f69b0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:59:47.433 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:47.430+0000 7fccd05fe700 1 -- 172.21.15.71:0/4042958904 >> 172.21.15.71:0/4042958904 conn(0x7fccc80f1cf0 msgr2=0x7fccc80f4110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:59:47.433 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:47.431+0000 7fccd05fe700 1 -- 172.21.15.71:0/4042958904 shutdown_connections 2024-02-20T12:59:47.433 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:47.431+0000 7fccd05fe700 1 -- 172.21.15.71:0/4042958904 wait complete. 2024-02-20T12:59:47.433 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:47.431+0000 7fccd05fe700 1 Processor -- start 2024-02-20T12:59:47.433 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:47.431+0000 7fccd05fe700 1 -- start start 2024-02-20T12:59:47.433 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:47.431+0000 7fccd05fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fccc80f6590 0x7fccc8106780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:59:47.433 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:47.431+0000 7fccd05fe700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccc8106c70 con 0x7fccc80f6590 2024-02-20T12:59:47.433 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:47.431+0000 7fccce39a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fccc80f6590 0x7fccc8106780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:59:47.433 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:47.431+0000 7fccce39a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fccc80f6590 0x7fccc8106780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.71:51902/0 (socket says 172.21.15.71:51902) 2024-02-20T12:59:47.433 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:47.431+0000 7fccce39a700 1 -- 172.21.15.71:0/4174962857 learned_addr learned my addr 172.21.15.71:0/4174962857 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-02-20T12:59:47.433 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:47.432+0000 7fccce39a700 1 -- 172.21.15.71:0/4174962857 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccb8005040 con 0x7fccc80f6590 2024-02-20T12:59:47.433 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:47.432+0000 7fccce39a700 1 --2- 172.21.15.71:0/4174962857 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fccc80f6590 0x7fccc8106780 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 rx=0x7fccb80089f0 tx=0x7fccb800b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:59:47.433 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:47.432+0000 7fccbeffd700 1 -- 172.21.15.71:0/4174962857 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fccb8009320 con 0x7fccc80f6590 2024-02-20T12:59:47.434 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:47.432+0000 7fccd05fe700 1 -- 172.21.15.71:0/4174962857 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fccc8106e70 con 0x7fccc80f6590 2024-02-20T12:59:47.434 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:47.432+0000 7fccbeffd700 1 -- 172.21.15.71:0/4174962857 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(25 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7fccb8013ec0 con 0x7fccc80f6590 2024-02-20T12:59:47.434 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:47.432+0000 7fccbeffd700 1 -- 172.21.15.71:0/4174962857 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fccb8014d30 con 0x7fccc80f6590 2024-02-20T12:59:47.434 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:47.432+0000 7fccd05fe700 1 -- 172.21.15.71:0/4174962857 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fccc8107300 con 0x7fccc80f6590 2024-02-20T12:59:47.435 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:47.433+0000 7fccbeffd700 1 -- 172.21.15.71:0/4174962857 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 37151+0+0 (secure 0 0 0) 0x7fccb80095f0 con 0x7fccc80f6590 2024-02-20T12:59:47.435 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:47.433+0000 7fccd05fe700 1 -- 172.21.15.71:0/4174962857 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fccc8045bb0 con 0x7fccc80f6590 2024-02-20T12:59:47.435 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:47.433+0000 7fccbeffd700 1 --2- 172.21.15.71:0/4174962857 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fccb4031090 0x7fccb4033550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:59:47.435 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:47.433+0000 7fccbeffd700 1 -- 172.21.15.71:0/4174962857 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1065+0+0 (secure 0 0 0) 0x7fccb80423e0 con 0x7fccc80f6590 2024-02-20T12:59:47.435 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:47.433+0000 7fcccdb99700 1 --2- 172.21.15.71:0/4174962857 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fccb4031090 0x7fccb4033550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:59:47.435 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:47.434+0000 7fcccdb99700 1 --2- 172.21.15.71:0/4174962857 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fccb4031090 0x7fccb4033550 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 rx=0x7fccc40096a0 tx=0x7fccc4006b40).ready entity=mgr.14156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:59:47.438 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:47.438+0000 7fccbeffd700 1 -- 172.21.15.71:0/4174962857 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+153237 (secure 0 0 0) 0x7fccb801d7f0 con 0x7fccc80f6590 2024-02-20T12:59:47.648 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:47.647+0000 7fccd05fe700 1 -- 172.21.15.71:0/4174962857 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fccc81081b0 con 0x7fccc80f6590 2024-02-20T12:59:47.648 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:47.648+0000 7fccbeffd700 1 -- 172.21.15.71:0/4174962857 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7fccb801d7f0 con 0x7fccc80f6590 2024-02-20T12:59:47.649 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-20T12:59:47.649 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":1,"fsid":"680ee126-cfee-11ee-95bb-87774f69a715","modified":"2024-02-20T12:49:44.699059Z","created":"2024-02-20T12:49:44.699059Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi064","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:3300","nonce":0},{"type":"v1","addr":"172.21.15.64:6789","nonce":0}]},"addr":"172.21.15.64:6789/0","public_addr":"172.21.15.64:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-20T12:59:47.650 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:47.650+0000 7fccd05fe700 1 -- 172.21.15.71:0/4174962857 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fccb4031090 msgr2=0x7fccb4033550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:59:47.650 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:47.650+0000 7fccd05fe700 1 --2- 172.21.15.71:0/4174962857 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fccb4031090 0x7fccb4033550 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 rx=0x7fccc40096a0 tx=0x7fccc4006b40).stop 2024-02-20T12:59:47.650 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:47.650+0000 7fccd05fe700 1 -- 172.21.15.71:0/4174962857 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fccc80f6590 msgr2=0x7fccc8106780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:59:47.651 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:47.650+0000 7fccd05fe700 1 --2- 172.21.15.71:0/4174962857 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fccc80f6590 0x7fccc8106780 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 rx=0x7fccb80089f0 tx=0x7fccb800b040).stop 2024-02-20T12:59:47.651 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:47.650+0000 7fccd05fe700 1 -- 172.21.15.71:0/4174962857 shutdown_connections 2024-02-20T12:59:47.651 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:47.650+0000 7fccd05fe700 1 --2- 172.21.15.71:0/4174962857 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fccb4031090 0x7fccb4033550 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:59:47.651 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:47.650+0000 7fccd05fe700 1 --2- 172.21.15.71:0/4174962857 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fccc80f6590 0x7fccc8106780 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:59:47.651 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:47.650+0000 7fccd05fe700 1 -- 172.21.15.71:0/4174962857 >> 172.21.15.71:0/4174962857 conn(0x7fccc80f1cf0 msgr2=0x7fccc80f27e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:59:47.651 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:47.650+0000 7fccd05fe700 1 -- 172.21.15.71:0/4174962857 shutdown_connections 2024-02-20T12:59:47.651 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:47.650+0000 7fccd05fe700 1 -- 172.21.15.71:0/4174962857 wait complete. 2024-02-20T12:59:47.651 INFO:teuthology.orchestra.run.smithi071.stderr:dumped monmap epoch 1 2024-02-20T12:59:48.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:59:48 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:59:47.649059+0000 mon.smithi064 (mon.0) 265 : audit [DBG] from='client.? 172.21.15.71:0/4174962857' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-20T12:59:49.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:59:49 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:59:47.791464+0000 mgr.smithi064.vriyor (mgr.14156) 278 : cluster [DBG] pgmap v261: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:59:49.391 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-20T12:59:49.391 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/daemon-base:latest-pacific shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 680ee126-cfee-11ee-95bb-87774f69a715 -- ceph mon dump -f json 2024-02-20T12:59:51.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:59:51 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:59:49.791735+0000 mgr.smithi064.vriyor (mgr.14156) 279 : cluster [DBG] pgmap v262: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:59:51.318 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:51.316+0000 7ff8c1ef1700 1 -- 172.21.15.71:0/3042254304 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff8bc0f5480 msgr2=0x7ff8bc0f58a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:59:51.318 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:51.316+0000 7ff8c1ef1700 1 --2- 172.21.15.71:0/3042254304 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff8bc0f5480 0x7ff8bc0f58a0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 rx=0x7ff8ac009fb0 tx=0x7ff8ac009b30).stop 2024-02-20T12:59:51.318 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:51.316+0000 7ff8c1ef1700 1 -- 172.21.15.71:0/3042254304 shutdown_connections 2024-02-20T12:59:51.318 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:51.316+0000 7ff8c1ef1700 1 --2- 172.21.15.71:0/3042254304 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff8bc0f5480 0x7ff8bc0f58a0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:59:51.319 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:51.316+0000 7ff8c1ef1700 1 -- 172.21.15.71:0/3042254304 >> 172.21.15.71:0/3042254304 conn(0x7ff8bc0f0be0 msgr2=0x7ff8bc0f3000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:59:51.319 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:51.316+0000 7ff8c1ef1700 1 -- 172.21.15.71:0/3042254304 shutdown_connections 2024-02-20T12:59:51.319 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:51.316+0000 7ff8c1ef1700 1 -- 172.21.15.71:0/3042254304 wait complete. 2024-02-20T12:59:51.319 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:51.317+0000 7ff8c1ef1700 1 Processor -- start 2024-02-20T12:59:51.319 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:51.317+0000 7ff8c1ef1700 1 -- start start 2024-02-20T12:59:51.319 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:51.317+0000 7ff8c1ef1700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff8bc0f5480 0x7ff8bc109510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:59:51.319 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:51.317+0000 7ff8c1ef1700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8bc060300 con 0x7ff8bc0f5480 2024-02-20T12:59:51.319 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:51.317+0000 7ff8bb7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff8bc0f5480 0x7ff8bc109510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:59:51.319 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:51.317+0000 7ff8bb7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff8bc0f5480 0x7ff8bc109510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.71:51914/0 (socket says 172.21.15.71:51914) 2024-02-20T12:59:51.319 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:51.317+0000 7ff8bb7fe700 1 -- 172.21.15.71:0/3270801937 learned_addr learned my addr 172.21.15.71:0/3270801937 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-02-20T12:59:51.319 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:51.317+0000 7ff8bb7fe700 1 -- 172.21.15.71:0/3270801937 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8ac005040 con 0x7ff8bc0f5480 2024-02-20T12:59:51.319 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:51.318+0000 7ff8bb7fe700 1 --2- 172.21.15.71:0/3270801937 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff8bc0f5480 0x7ff8bc109510 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 rx=0x7ff8ac0089f0 tx=0x7ff8ac00b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:59:51.319 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:51.318+0000 7ff8b8ff9700 1 -- 172.21.15.71:0/3270801937 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff8ac009320 con 0x7ff8bc0f5480 2024-02-20T12:59:51.319 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:51.318+0000 7ff8c1ef1700 1 -- 172.21.15.71:0/3270801937 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8bc060500 con 0x7ff8bc0f5480 2024-02-20T12:59:51.319 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:51.318+0000 7ff8b8ff9700 1 -- 172.21.15.71:0/3270801937 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(25 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7ff8ac013ec0 con 0x7ff8bc0f5480 2024-02-20T12:59:51.320 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:51.318+0000 7ff8b8ff9700 1 -- 172.21.15.71:0/3270801937 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff8ac014d30 con 0x7ff8bc0f5480 2024-02-20T12:59:51.320 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:51.318+0000 7ff8c1ef1700 1 -- 172.21.15.71:0/3270801937 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8bc109bc0 con 0x7ff8bc0f5480 2024-02-20T12:59:51.321 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:51.319+0000 7ff8b8ff9700 1 -- 172.21.15.71:0/3270801937 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 37151+0+0 (secure 0 0 0) 0x7ff8ac0095f0 con 0x7ff8bc0f5480 2024-02-20T12:59:51.321 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:51.319+0000 7ff8c1ef1700 1 -- 172.21.15.71:0/3270801937 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8bc0591e0 con 0x7ff8bc0f5480 2024-02-20T12:59:51.321 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:51.319+0000 7ff8b8ff9700 1 --2- 172.21.15.71:0/3270801937 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7ff8a8031090 0x7ff8a8033550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:59:51.321 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:51.319+0000 7ff8b8ff9700 1 -- 172.21.15.71:0/3270801937 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1065+0+0 (secure 0 0 0) 0x7ff8ac0423e0 con 0x7ff8bc0f5480 2024-02-20T12:59:51.321 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:51.319+0000 7ff8b3fff700 1 --2- 172.21.15.71:0/3270801937 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7ff8a8031090 0x7ff8a8033550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:59:51.321 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:51.319+0000 7ff8b3fff700 1 --2- 172.21.15.71:0/3270801937 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7ff8a8031090 0x7ff8a8033550 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 rx=0x7ff8a40096a0 tx=0x7ff8a4006b40).ready entity=mgr.14156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:59:51.324 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:51.324+0000 7ff8b8ff9700 1 -- 172.21.15.71:0/3270801937 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+153237 (secure 0 0 0) 0x7ff8ac01db60 con 0x7ff8bc0f5480 2024-02-20T12:59:51.530 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:51.530+0000 7ff8c1ef1700 1 -- 172.21.15.71:0/3270801937 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7ff8bc045bb0 con 0x7ff8bc0f5480 2024-02-20T12:59:51.531 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:51.530+0000 7ff8b8ff9700 1 -- 172.21.15.71:0/3270801937 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7ff8ac01d380 con 0x7ff8bc0f5480 2024-02-20T12:59:51.531 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-20T12:59:51.531 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":1,"fsid":"680ee126-cfee-11ee-95bb-87774f69a715","modified":"2024-02-20T12:49:44.699059Z","created":"2024-02-20T12:49:44.699059Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi064","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:3300","nonce":0},{"type":"v1","addr":"172.21.15.64:6789","nonce":0}]},"addr":"172.21.15.64:6789/0","public_addr":"172.21.15.64:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-20T12:59:51.532 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:51.532+0000 7ff8c1ef1700 1 -- 172.21.15.71:0/3270801937 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7ff8a8031090 msgr2=0x7ff8a8033550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:59:51.532 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:51.532+0000 7ff8c1ef1700 1 --2- 172.21.15.71:0/3270801937 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7ff8a8031090 0x7ff8a8033550 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 rx=0x7ff8a40096a0 tx=0x7ff8a4006b40).stop 2024-02-20T12:59:51.533 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:51.532+0000 7ff8c1ef1700 1 -- 172.21.15.71:0/3270801937 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff8bc0f5480 msgr2=0x7ff8bc109510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:59:51.533 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:51.532+0000 7ff8c1ef1700 1 --2- 172.21.15.71:0/3270801937 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff8bc0f5480 0x7ff8bc109510 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 rx=0x7ff8ac0089f0 tx=0x7ff8ac00b040).stop 2024-02-20T12:59:51.533 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:51.532+0000 7ff8c1ef1700 1 -- 172.21.15.71:0/3270801937 shutdown_connections 2024-02-20T12:59:51.533 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:51.532+0000 7ff8c1ef1700 1 --2- 172.21.15.71:0/3270801937 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7ff8a8031090 0x7ff8a8033550 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:59:51.533 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:51.532+0000 7ff8c1ef1700 1 --2- 172.21.15.71:0/3270801937 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff8bc0f5480 0x7ff8bc109510 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:59:51.533 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:51.532+0000 7ff8c1ef1700 1 -- 172.21.15.71:0/3270801937 >> 172.21.15.71:0/3270801937 conn(0x7ff8bc0f0be0 msgr2=0x7ff8bc0f16d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:59:51.533 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:51.532+0000 7ff8c1ef1700 1 -- 172.21.15.71:0/3270801937 shutdown_connections 2024-02-20T12:59:51.533 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:51.532+0000 7ff8c1ef1700 1 -- 172.21.15.71:0/3270801937 wait complete. 2024-02-20T12:59:51.533 INFO:teuthology.orchestra.run.smithi071.stderr:dumped monmap epoch 1 2024-02-20T12:59:52.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:59:52 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:59:51.531104+0000 mon.smithi064 (mon.0) 266 : audit [DBG] from='client.? 172.21.15.71:0/3270801937' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-20T12:59:53.142 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-20T12:59:53.143 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/daemon-base:latest-pacific shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 680ee126-cfee-11ee-95bb-87774f69a715 -- ceph mon dump -f json 2024-02-20T12:59:53.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:59:53 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:59:51.791950+0000 mgr.smithi064.vriyor (mgr.14156) 280 : cluster [DBG] pgmap v263: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:59:54.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:59:54 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:59:53.851837+0000 mon.smithi064 (mon.0) 267 : audit [INF] from='mgr.14156 172.21.15.64:0/3284974785' entity='mgr.smithi064.vriyor' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi064.vriyor/mirror_snapshot_schedule"}]: dispatch 2024-02-20T12:59:54.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:59:54 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:59:53.852085+0000 mon.smithi064 (mon.0) 268 : audit [INF] from='mgr.14156 172.21.15.64:0/3284974785' entity='mgr.smithi064.vriyor' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi064.vriyor/trash_purge_schedule"}]: dispatch 2024-02-20T12:59:55.102 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:55.093+0000 7fcb25d60700 1 -- 172.21.15.71:0/4115878046 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcb20060360 msgr2=0x7fcb200f9160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:59:55.102 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:55.093+0000 7fcb25d60700 1 --2- 172.21.15.71:0/4115878046 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcb20060360 0x7fcb200f9160 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 rx=0x7fcb10009960 tx=0x7fcb10004d10).stop 2024-02-20T12:59:55.102 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:55.093+0000 7fcb25d60700 1 -- 172.21.15.71:0/4115878046 shutdown_connections 2024-02-20T12:59:55.103 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:55.093+0000 7fcb25d60700 1 --2- 172.21.15.71:0/4115878046 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcb20060360 0x7fcb200f9160 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:59:55.103 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:55.093+0000 7fcb25d60700 1 -- 172.21.15.71:0/4115878046 >> 172.21.15.71:0/4115878046 conn(0x7fcb200f0be0 msgr2=0x7fcb200f3000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:59:55.103 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:55.093+0000 7fcb25d60700 1 -- 172.21.15.71:0/4115878046 shutdown_connections 2024-02-20T12:59:55.103 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:55.093+0000 7fcb25d60700 1 -- 172.21.15.71:0/4115878046 wait complete. 2024-02-20T12:59:55.103 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:55.094+0000 7fcb25d60700 1 Processor -- start 2024-02-20T12:59:55.103 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:55.094+0000 7fcb25d60700 1 -- start start 2024-02-20T12:59:55.103 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:55.094+0000 7fcb25d60700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcb20060360 0x7fcb20106750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:59:55.103 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:55.094+0000 7fcb25d60700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb20106c40 con 0x7fcb20060360 2024-02-20T12:59:55.103 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:55.094+0000 7fcb1f7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcb20060360 0x7fcb20106750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:59:55.103 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:55.094+0000 7fcb1f7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcb20060360 0x7fcb20106750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.71:51920/0 (socket says 172.21.15.71:51920) 2024-02-20T12:59:55.103 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:55.094+0000 7fcb1f7fe700 1 -- 172.21.15.71:0/1662273765 learned_addr learned my addr 172.21.15.71:0/1662273765 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-02-20T12:59:55.103 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:55.095+0000 7fcb1f7fe700 1 -- 172.21.15.71:0/1662273765 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb10009640 con 0x7fcb20060360 2024-02-20T12:59:55.103 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:55.095+0000 7fcb1f7fe700 1 --2- 172.21.15.71:0/1662273765 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcb20060360 0x7fcb20106750 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 rx=0x7fcb10009610 tx=0x7fcb10004d10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:59:55.103 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:55.095+0000 7fcb17fff700 1 -- 172.21.15.71:0/1662273765 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fcb10018070 con 0x7fcb20060360 2024-02-20T12:59:55.104 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:55.095+0000 7fcb25d60700 1 -- 172.21.15.71:0/1662273765 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb20106e40 con 0x7fcb20060360 2024-02-20T12:59:55.104 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:55.095+0000 7fcb17fff700 1 -- 172.21.15.71:0/1662273765 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(25 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7fcb1000b7e0 con 0x7fcb20060360 2024-02-20T12:59:55.104 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:55.095+0000 7fcb17fff700 1 -- 172.21.15.71:0/1662273765 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fcb1001d420 con 0x7fcb20060360 2024-02-20T12:59:55.104 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:55.095+0000 7fcb25d60700 1 -- 172.21.15.71:0/1662273765 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb201072d0 con 0x7fcb20060360 2024-02-20T12:59:55.104 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:55.096+0000 7fcb17fff700 1 -- 172.21.15.71:0/1662273765 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 37151+0+0 (secure 0 0 0) 0x7fcb10024060 con 0x7fcb20060360 2024-02-20T12:59:55.104 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:55.096+0000 7fcb25d60700 1 -- 172.21.15.71:0/1662273765 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb20045bb0 con 0x7fcb20060360 2024-02-20T12:59:55.104 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:55.096+0000 7fcb17fff700 1 --2- 172.21.15.71:0/1662273765 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fcb0c0310e0 0x7fcb0c0335a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:59:55.104 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:55.096+0000 7fcb17fff700 1 -- 172.21.15.71:0/1662273765 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1065+0+0 (secure 0 0 0) 0x7fcb100426e0 con 0x7fcb20060360 2024-02-20T12:59:55.104 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:55.096+0000 7fcb1effd700 1 --2- 172.21.15.71:0/1662273765 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fcb0c0310e0 0x7fcb0c0335a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:59:55.104 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:55.097+0000 7fcb1effd700 1 --2- 172.21.15.71:0/1662273765 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fcb0c0310e0 0x7fcb0c0335a0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 rx=0x7fcb080096a0 tx=0x7fcb08006b40).ready entity=mgr.14156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:59:55.104 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:55.101+0000 7fcb17fff700 1 -- 172.21.15.71:0/1662273765 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+153237 (secure 0 0 0) 0x7fcb10011b60 con 0x7fcb20060360 2024-02-20T12:59:55.308 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:55.307+0000 7fcb25d60700 1 -- 172.21.15.71:0/1662273765 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fcb20108120 con 0x7fcb20060360 2024-02-20T12:59:55.308 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:55.308+0000 7fcb17fff700 1 -- 172.21.15.71:0/1662273765 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7fcb10011930 con 0x7fcb20060360 2024-02-20T12:59:55.309 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-20T12:59:55.309 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":1,"fsid":"680ee126-cfee-11ee-95bb-87774f69a715","modified":"2024-02-20T12:49:44.699059Z","created":"2024-02-20T12:49:44.699059Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi064","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:3300","nonce":0},{"type":"v1","addr":"172.21.15.64:6789","nonce":0}]},"addr":"172.21.15.64:6789/0","public_addr":"172.21.15.64:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-20T12:59:55.310 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:55.309+0000 7fcb25d60700 1 -- 172.21.15.71:0/1662273765 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fcb0c0310e0 msgr2=0x7fcb0c0335a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:59:55.310 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:55.309+0000 7fcb25d60700 1 --2- 172.21.15.71:0/1662273765 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fcb0c0310e0 0x7fcb0c0335a0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 rx=0x7fcb080096a0 tx=0x7fcb08006b40).stop 2024-02-20T12:59:55.310 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:55.309+0000 7fcb25d60700 1 -- 172.21.15.71:0/1662273765 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcb20060360 msgr2=0x7fcb20106750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:59:55.310 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:55.309+0000 7fcb25d60700 1 --2- 172.21.15.71:0/1662273765 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcb20060360 0x7fcb20106750 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 rx=0x7fcb10009610 tx=0x7fcb10004d10).stop 2024-02-20T12:59:55.310 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:55.310+0000 7fcb25d60700 1 -- 172.21.15.71:0/1662273765 shutdown_connections 2024-02-20T12:59:55.310 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:55.310+0000 7fcb25d60700 1 --2- 172.21.15.71:0/1662273765 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fcb0c0310e0 0x7fcb0c0335a0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:59:55.310 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:55.310+0000 7fcb25d60700 1 --2- 172.21.15.71:0/1662273765 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcb20060360 0x7fcb20106750 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:59:55.310 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:55.310+0000 7fcb25d60700 1 -- 172.21.15.71:0/1662273765 >> 172.21.15.71:0/1662273765 conn(0x7fcb200f0be0 msgr2=0x7fcb200f1660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:59:55.310 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:55.310+0000 7fcb25d60700 1 -- 172.21.15.71:0/1662273765 shutdown_connections 2024-02-20T12:59:55.311 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:55.310+0000 7fcb25d60700 1 -- 172.21.15.71:0/1662273765 wait complete. 2024-02-20T12:59:55.311 INFO:teuthology.orchestra.run.smithi071.stderr:dumped monmap epoch 1 2024-02-20T12:59:55.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:59:55 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:59:53.792208+0000 mgr.smithi064.vriyor (mgr.14156) 281 : cluster [DBG] pgmap v264: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:59:56.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:59:56 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:59:55.308876+0000 mon.smithi064 (mon.0) 269 : audit [DBG] from='client.? 172.21.15.71:0/1662273765' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-20T12:59:56.965 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-20T12:59:56.965 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/daemon-base:latest-pacific shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 680ee126-cfee-11ee-95bb-87774f69a715 -- ceph mon dump -f json 2024-02-20T12:59:57.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:59:57 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:59:55.792592+0000 mgr.smithi064.vriyor (mgr.14156) 282 : cluster [DBG] pgmap v265: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T12:59:58.922 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:58.917+0000 7fbc84b9f700 1 -- 172.21.15.71:0/2085119727 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbc800f63e0 msgr2=0x7fbc800f6800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:59:58.922 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:58.917+0000 7fbc84b9f700 1 --2- 172.21.15.71:0/2085119727 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbc800f63e0 0x7fbc800f6800 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 rx=0x7fbc74009fb0 tx=0x7fbc74009b30).stop 2024-02-20T12:59:58.922 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:58.917+0000 7fbc84b9f700 1 -- 172.21.15.71:0/2085119727 shutdown_connections 2024-02-20T12:59:58.922 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:58.917+0000 7fbc84b9f700 1 --2- 172.21.15.71:0/2085119727 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbc800f63e0 0x7fbc800f6800 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:59:58.922 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:58.917+0000 7fbc84b9f700 1 -- 172.21.15.71:0/2085119727 >> 172.21.15.71:0/2085119727 conn(0x7fbc800f1b40 msgr2=0x7fbc800f3f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:59:58.923 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:58.917+0000 7fbc84b9f700 1 -- 172.21.15.71:0/2085119727 shutdown_connections 2024-02-20T12:59:58.923 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:58.917+0000 7fbc84b9f700 1 -- 172.21.15.71:0/2085119727 wait complete. 2024-02-20T12:59:58.923 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:58.918+0000 7fbc84b9f700 1 Processor -- start 2024-02-20T12:59:58.923 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:58.918+0000 7fbc84b9f700 1 -- start start 2024-02-20T12:59:58.923 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:58.918+0000 7fbc84b9f700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbc800f63e0 0x7fbc80106620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:59:58.923 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:58.918+0000 7fbc84b9f700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc80106b10 con 0x7fbc800f63e0 2024-02-20T12:59:58.924 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:58.918+0000 7fbc7f7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbc800f63e0 0x7fbc80106620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:59:58.925 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:58.918+0000 7fbc7f7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbc800f63e0 0x7fbc80106620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.71:41420/0 (socket says 172.21.15.71:41420) 2024-02-20T12:59:58.925 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:58.918+0000 7fbc7f7fe700 1 -- 172.21.15.71:0/1787783009 learned_addr learned my addr 172.21.15.71:0/1787783009 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-02-20T12:59:58.925 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:58.918+0000 7fbc7f7fe700 1 -- 172.21.15.71:0/1787783009 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc74005040 con 0x7fbc800f63e0 2024-02-20T12:59:58.925 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:58.918+0000 7fbc7f7fe700 1 --2- 172.21.15.71:0/1787783009 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbc800f63e0 0x7fbc80106620 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 rx=0x7fbc740089f0 tx=0x7fbc7400b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:59:58.925 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:58.919+0000 7fbc67fff700 1 -- 172.21.15.71:0/1787783009 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fbc74009320 con 0x7fbc800f63e0 2024-02-20T12:59:58.925 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:58.919+0000 7fbc84b9f700 1 -- 172.21.15.71:0/1787783009 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc80106d10 con 0x7fbc800f63e0 2024-02-20T12:59:58.925 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:58.919+0000 7fbc67fff700 1 -- 172.21.15.71:0/1787783009 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(25 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7fbc74013ec0 con 0x7fbc800f63e0 2024-02-20T12:59:58.925 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:58.919+0000 7fbc67fff700 1 -- 172.21.15.71:0/1787783009 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fbc74014d30 con 0x7fbc800f63e0 2024-02-20T12:59:58.925 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:58.919+0000 7fbc84b9f700 1 -- 172.21.15.71:0/1787783009 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc801071a0 con 0x7fbc800f63e0 2024-02-20T12:59:58.925 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:58.920+0000 7fbc67fff700 1 -- 172.21.15.71:0/1787783009 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 37151+0+0 (secure 0 0 0) 0x7fbc740095f0 con 0x7fbc800f63e0 2024-02-20T12:59:58.925 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:58.920+0000 7fbc84b9f700 1 -- 172.21.15.71:0/1787783009 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc80045bb0 con 0x7fbc800f63e0 2024-02-20T12:59:58.925 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:58.920+0000 7fbc67fff700 1 --2- 172.21.15.71:0/1787783009 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fbc68031090 0x7fbc68033550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T12:59:58.925 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:58.920+0000 7fbc67fff700 1 -- 172.21.15.71:0/1787783009 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1065+0+0 (secure 0 0 0) 0x7fbc740424a0 con 0x7fbc800f63e0 2024-02-20T12:59:58.925 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:58.920+0000 7fbc7effd700 1 --2- 172.21.15.71:0/1787783009 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fbc68031090 0x7fbc68033550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T12:59:58.925 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:58.920+0000 7fbc7effd700 1 --2- 172.21.15.71:0/1787783009 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fbc68031090 0x7fbc68033550 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 rx=0x7fbc700096a0 tx=0x7fbc70006b40).ready entity=mgr.14156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T12:59:58.926 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:58.925+0000 7fbc67fff700 1 -- 172.21.15.71:0/1787783009 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+153237 (secure 0 0 0) 0x7fbc7401d980 con 0x7fbc800f63e0 2024-02-20T12:59:59.131 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:59.130+0000 7fbc84b9f700 1 -- 172.21.15.71:0/1787783009 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fbc80108050 con 0x7fbc800f63e0 2024-02-20T12:59:59.131 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:59.131+0000 7fbc67fff700 1 -- 172.21.15.71:0/1787783009 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7fbc7401d380 con 0x7fbc800f63e0 2024-02-20T12:59:59.132 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-20T12:59:59.132 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":1,"fsid":"680ee126-cfee-11ee-95bb-87774f69a715","modified":"2024-02-20T12:49:44.699059Z","created":"2024-02-20T12:49:44.699059Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi064","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:3300","nonce":0},{"type":"v1","addr":"172.21.15.64:6789","nonce":0}]},"addr":"172.21.15.64:6789/0","public_addr":"172.21.15.64:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-20T12:59:59.133 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:59.132+0000 7fbc84b9f700 1 -- 172.21.15.71:0/1787783009 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fbc68031090 msgr2=0x7fbc68033550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:59:59.133 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:59.132+0000 7fbc84b9f700 1 --2- 172.21.15.71:0/1787783009 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fbc68031090 0x7fbc68033550 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 rx=0x7fbc700096a0 tx=0x7fbc70006b40).stop 2024-02-20T12:59:59.133 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:59.132+0000 7fbc84b9f700 1 -- 172.21.15.71:0/1787783009 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbc800f63e0 msgr2=0x7fbc80106620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T12:59:59.133 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:59.132+0000 7fbc84b9f700 1 --2- 172.21.15.71:0/1787783009 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbc800f63e0 0x7fbc80106620 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 rx=0x7fbc740089f0 tx=0x7fbc7400b040).stop 2024-02-20T12:59:59.133 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:59.132+0000 7fbc84b9f700 1 -- 172.21.15.71:0/1787783009 shutdown_connections 2024-02-20T12:59:59.133 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:59.132+0000 7fbc84b9f700 1 --2- 172.21.15.71:0/1787783009 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fbc68031090 0x7fbc68033550 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:59:59.133 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:59.132+0000 7fbc84b9f700 1 --2- 172.21.15.71:0/1787783009 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbc800f63e0 0x7fbc80106620 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T12:59:59.133 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:59.132+0000 7fbc84b9f700 1 -- 172.21.15.71:0/1787783009 >> 172.21.15.71:0/1787783009 conn(0x7fbc800f1b40 msgr2=0x7fbc800f26a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T12:59:59.134 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:59.133+0000 7fbc84b9f700 1 -- 172.21.15.71:0/1787783009 shutdown_connections 2024-02-20T12:59:59.134 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T12:59:59.133+0000 7fbc84b9f700 1 -- 172.21.15.71:0/1787783009 wait complete. 2024-02-20T12:59:59.134 INFO:teuthology.orchestra.run.smithi071.stderr:dumped monmap epoch 1 2024-02-20T12:59:59.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 12:59:59 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:59:57.792929+0000 mgr.smithi064.vriyor (mgr.14156) 283 : cluster [DBG] pgmap v266: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T13:00:00.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 13:00:00 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T12:59:59.131745+0000 mon.smithi064 (mon.0) 270 : audit [DBG] from='client.? 172.21.15.71:0/1787783009' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-20T13:00:00.760 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-20T13:00:00.761 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/daemon-base:latest-pacific shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 680ee126-cfee-11ee-95bb-87774f69a715 -- ceph mon dump -f json 2024-02-20T13:00:01.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 13:00:01 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T12:59:59.793252+0000 mgr.smithi064.vriyor (mgr.14156) 284 : cluster [DBG] pgmap v267: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T13:00:02.793 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:02.793+0000 7f99216d1700 1 -- 172.21.15.71:0/2612990047 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f991c0f54b0 msgr2=0x7f991c0f58d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T13:00:02.793 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:02.793+0000 7f99216d1700 1 --2- 172.21.15.71:0/2612990047 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f991c0f54b0 0x7f991c0f58d0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 rx=0x7f990c009fb0 tx=0x7f990c009b30).stop 2024-02-20T13:00:02.793 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:02.793+0000 7f99216d1700 1 -- 172.21.15.71:0/2612990047 shutdown_connections 2024-02-20T13:00:02.794 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:02.793+0000 7f99216d1700 1 --2- 172.21.15.71:0/2612990047 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f991c0f54b0 0x7f991c0f58d0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T13:00:02.794 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:02.793+0000 7f99216d1700 1 -- 172.21.15.71:0/2612990047 >> 172.21.15.71:0/2612990047 conn(0x7f991c0f0c10 msgr2=0x7f991c0f3030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T13:00:02.794 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:02.793+0000 7f99216d1700 1 -- 172.21.15.71:0/2612990047 shutdown_connections 2024-02-20T13:00:02.794 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:02.793+0000 7f99216d1700 1 -- 172.21.15.71:0/2612990047 wait complete. 2024-02-20T13:00:02.797 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:02.793+0000 7f99216d1700 1 Processor -- start 2024-02-20T13:00:02.797 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:02.793+0000 7f99216d1700 1 -- start start 2024-02-20T13:00:02.797 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:02.794+0000 7f99216d1700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f991c0f54b0 0x7f991c1094a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T13:00:02.797 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:02.794+0000 7f99216d1700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f991c060320 con 0x7f991c0f54b0 2024-02-20T13:00:02.797 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:02.794+0000 7f991affd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f991c0f54b0 0x7f991c1094a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T13:00:02.797 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:02.794+0000 7f991affd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f991c0f54b0 0x7f991c1094a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.71:41432/0 (socket says 172.21.15.71:41432) 2024-02-20T13:00:02.797 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:02.794+0000 7f991affd700 1 -- 172.21.15.71:0/1717550549 learned_addr learned my addr 172.21.15.71:0/1717550549 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-02-20T13:00:02.797 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:02.794+0000 7f991affd700 1 -- 172.21.15.71:0/1717550549 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f990c005040 con 0x7f991c0f54b0 2024-02-20T13:00:02.797 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:02.794+0000 7f991affd700 1 --2- 172.21.15.71:0/1717550549 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f991c0f54b0 0x7f991c1094a0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 rx=0x7f990c0089f0 tx=0x7f990c00b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T13:00:02.797 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:02.794+0000 7f99037fe700 1 -- 172.21.15.71:0/1717550549 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f990c009320 con 0x7f991c0f54b0 2024-02-20T13:00:02.797 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:02.794+0000 7f99037fe700 1 -- 172.21.15.71:0/1717550549 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(25 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f990c013ec0 con 0x7f991c0f54b0 2024-02-20T13:00:02.797 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:02.794+0000 7f99216d1700 1 -- 172.21.15.71:0/1717550549 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f991c060520 con 0x7f991c0f54b0 2024-02-20T13:00:02.798 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:02.794+0000 7f99037fe700 1 -- 172.21.15.71:0/1717550549 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f990c014d30 con 0x7f991c0f54b0 2024-02-20T13:00:02.798 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:02.794+0000 7f99216d1700 1 -- 172.21.15.71:0/1717550549 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f991c109b50 con 0x7f991c0f54b0 2024-02-20T13:00:02.798 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:02.795+0000 7f99037fe700 1 -- 172.21.15.71:0/1717550549 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 37151+0+0 (secure 0 0 0) 0x7f990c0095f0 con 0x7f991c0f54b0 2024-02-20T13:00:02.798 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:02.795+0000 7f99037fe700 1 --2- 172.21.15.71:0/1717550549 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f99040310e0 0x7f99040335a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T13:00:02.798 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:02.795+0000 7f99216d1700 1 -- 172.21.15.71:0/1717550549 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f991c045bb0 con 0x7f991c0f54b0 2024-02-20T13:00:02.798 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:02.795+0000 7f99037fe700 1 -- 172.21.15.71:0/1717550549 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1065+0+0 (secure 0 0 0) 0x7f990c0423a0 con 0x7f991c0f54b0 2024-02-20T13:00:02.798 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:02.796+0000 7f991a7fc700 1 --2- 172.21.15.71:0/1717550549 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f99040310e0 0x7f99040335a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T13:00:02.798 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:02.796+0000 7f991a7fc700 1 --2- 172.21.15.71:0/1717550549 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f99040310e0 0x7f99040335a0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 rx=0x7f99100096a0 tx=0x7f9910006b40).ready entity=mgr.14156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T13:00:02.801 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:02.800+0000 7f99037fe700 1 -- 172.21.15.71:0/1717550549 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+153237 (secure 0 0 0) 0x7f990c00fe40 con 0x7f991c0f54b0 2024-02-20T13:00:03.009 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:03.008+0000 7f99216d1700 1 -- 172.21.15.71:0/1717550549 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f991c10a370 con 0x7f991c0f54b0 2024-02-20T13:00:03.009 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:03.009+0000 7f99037fe700 1 -- 172.21.15.71:0/1717550549 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7f990c00fe40 con 0x7f991c0f54b0 2024-02-20T13:00:03.010 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-20T13:00:03.010 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":1,"fsid":"680ee126-cfee-11ee-95bb-87774f69a715","modified":"2024-02-20T12:49:44.699059Z","created":"2024-02-20T12:49:44.699059Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi064","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:3300","nonce":0},{"type":"v1","addr":"172.21.15.64:6789","nonce":0}]},"addr":"172.21.15.64:6789/0","public_addr":"172.21.15.64:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-20T13:00:03.011 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:03.011+0000 7f99216d1700 1 -- 172.21.15.71:0/1717550549 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f99040310e0 msgr2=0x7f99040335a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T13:00:03.011 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:03.011+0000 7f99216d1700 1 --2- 172.21.15.71:0/1717550549 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f99040310e0 0x7f99040335a0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 rx=0x7f99100096a0 tx=0x7f9910006b40).stop 2024-02-20T13:00:03.011 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:03.011+0000 7f99216d1700 1 -- 172.21.15.71:0/1717550549 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f991c0f54b0 msgr2=0x7f991c1094a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T13:00:03.011 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:03.011+0000 7f99216d1700 1 --2- 172.21.15.71:0/1717550549 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f991c0f54b0 0x7f991c1094a0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 rx=0x7f990c0089f0 tx=0x7f990c00b040).stop 2024-02-20T13:00:03.011 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:03.011+0000 7f99216d1700 1 -- 172.21.15.71:0/1717550549 shutdown_connections 2024-02-20T13:00:03.011 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:03.011+0000 7f99216d1700 1 --2- 172.21.15.71:0/1717550549 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f99040310e0 0x7f99040335a0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T13:00:03.011 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:03.011+0000 7f99216d1700 1 --2- 172.21.15.71:0/1717550549 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f991c0f54b0 0x7f991c1094a0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T13:00:03.012 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:03.011+0000 7f99216d1700 1 -- 172.21.15.71:0/1717550549 >> 172.21.15.71:0/1717550549 conn(0x7f991c0f0c10 msgr2=0x7f991c0f1670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T13:00:03.012 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:03.011+0000 7f99216d1700 1 -- 172.21.15.71:0/1717550549 shutdown_connections 2024-02-20T13:00:03.012 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:03.011+0000 7f99216d1700 1 -- 172.21.15.71:0/1717550549 wait complete. 2024-02-20T13:00:03.012 INFO:teuthology.orchestra.run.smithi071.stderr:dumped monmap epoch 1 2024-02-20T13:00:03.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 13:00:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T13:00:01.793628+0000 mgr.smithi064.vriyor (mgr.14156) 285 : cluster [DBG] pgmap v268: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T13:00:03.604 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 13:00:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T13:00:03.009935+0000 mon.smithi064 (mon.0) 271 : audit [DBG] from='client.? 172.21.15.71:0/1717550549' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-20T13:00:04.731 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-20T13:00:04.731 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/daemon-base:latest-pacific shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 680ee126-cfee-11ee-95bb-87774f69a715 -- ceph mon dump -f json 2024-02-20T13:00:05.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 13:00:05 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T13:00:03.793843+0000 mgr.smithi064.vriyor (mgr.14156) 286 : cluster [DBG] pgmap v269: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T13:00:06.623 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:06.618+0000 7fd280e54700 1 -- 172.21.15.71:0/3206113272 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd27c0f5480 msgr2=0x7fd27c0f58a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T13:00:06.623 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:06.618+0000 7fd280e54700 1 --2- 172.21.15.71:0/3206113272 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd27c0f5480 0x7fd27c0f58a0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 rx=0x7fd264009fb0 tx=0x7fd264009b30).stop 2024-02-20T13:00:06.623 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:06.619+0000 7fd280e54700 1 -- 172.21.15.71:0/3206113272 shutdown_connections 2024-02-20T13:00:06.623 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:06.619+0000 7fd280e54700 1 --2- 172.21.15.71:0/3206113272 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd27c0f5480 0x7fd27c0f58a0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T13:00:06.624 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:06.619+0000 7fd280e54700 1 -- 172.21.15.71:0/3206113272 >> 172.21.15.71:0/3206113272 conn(0x7fd27c0f0be0 msgr2=0x7fd27c0f3000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T13:00:06.624 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:06.619+0000 7fd280e54700 1 -- 172.21.15.71:0/3206113272 shutdown_connections 2024-02-20T13:00:06.624 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:06.619+0000 7fd280e54700 1 -- 172.21.15.71:0/3206113272 wait complete. 2024-02-20T13:00:06.624 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:06.619+0000 7fd280e54700 1 Processor -- start 2024-02-20T13:00:06.624 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:06.619+0000 7fd280e54700 1 -- start start 2024-02-20T13:00:06.625 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:06.619+0000 7fd280e54700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd27c0f5480 0x7fd27c1056f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T13:00:06.625 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:06.619+0000 7fd280e54700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd27c105be0 con 0x7fd27c0f5480 2024-02-20T13:00:06.625 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:06.619+0000 7fd27a59c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd27c0f5480 0x7fd27c1056f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T13:00:06.625 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:06.620+0000 7fd27a59c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd27c0f5480 0x7fd27c1056f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.71:41456/0 (socket says 172.21.15.71:41456) 2024-02-20T13:00:06.625 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:06.620+0000 7fd27a59c700 1 -- 172.21.15.71:0/1227659838 learned_addr learned my addr 172.21.15.71:0/1227659838 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-02-20T13:00:06.625 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:06.620+0000 7fd27a59c700 1 -- 172.21.15.71:0/1227659838 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd264005040 con 0x7fd27c0f5480 2024-02-20T13:00:06.626 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:06.620+0000 7fd27a59c700 1 --2- 172.21.15.71:0/1227659838 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd27c0f5480 0x7fd27c1056f0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 rx=0x7fd2640089f0 tx=0x7fd26400b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T13:00:06.626 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:06.620+0000 7fd272ffd700 1 -- 172.21.15.71:0/1227659838 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd264009320 con 0x7fd27c0f5480 2024-02-20T13:00:06.626 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:06.620+0000 7fd280e54700 1 -- 172.21.15.71:0/1227659838 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd27c105de0 con 0x7fd27c0f5480 2024-02-20T13:00:06.626 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:06.620+0000 7fd272ffd700 1 -- 172.21.15.71:0/1227659838 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(25 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7fd264013ec0 con 0x7fd27c0f5480 2024-02-20T13:00:06.626 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:06.620+0000 7fd272ffd700 1 -- 172.21.15.71:0/1227659838 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd264014d30 con 0x7fd27c0f5480 2024-02-20T13:00:06.626 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:06.620+0000 7fd280e54700 1 -- 172.21.15.71:0/1227659838 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd27c106270 con 0x7fd27c0f5480 2024-02-20T13:00:06.626 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:06.621+0000 7fd272ffd700 1 -- 172.21.15.71:0/1227659838 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 37151+0+0 (secure 0 0 0) 0x7fd2640095f0 con 0x7fd27c0f5480 2024-02-20T13:00:06.627 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:06.621+0000 7fd280e54700 1 -- 172.21.15.71:0/1227659838 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd27c045bb0 con 0x7fd27c0f5480 2024-02-20T13:00:06.627 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:06.621+0000 7fd272ffd700 1 --2- 172.21.15.71:0/1227659838 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fd268031090 0x7fd268033550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T13:00:06.627 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:06.621+0000 7fd272ffd700 1 -- 172.21.15.71:0/1227659838 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1065+0+0 (secure 0 0 0) 0x7fd264044240 con 0x7fd27c0f5480 2024-02-20T13:00:06.627 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:06.621+0000 7fd279d9b700 1 --2- 172.21.15.71:0/1227659838 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fd268031090 0x7fd268033550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T13:00:06.627 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:06.622+0000 7fd279d9b700 1 --2- 172.21.15.71:0/1227659838 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fd268031090 0x7fd268033550 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 rx=0x7fd26c0096a0 tx=0x7fd26c006b40).ready entity=mgr.14156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T13:00:06.628 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:06.626+0000 7fd272ffd700 1 -- 172.21.15.71:0/1227659838 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+153237 (secure 0 0 0) 0x7fd26401f980 con 0x7fd27c0f5480 2024-02-20T13:00:06.833 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:06.832+0000 7fd280e54700 1 -- 172.21.15.71:0/1227659838 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fd27c0faf70 con 0x7fd27c0f5480 2024-02-20T13:00:06.833 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:06.833+0000 7fd272ffd700 1 -- 172.21.15.71:0/1227659838 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7fd26401f380 con 0x7fd27c0f5480 2024-02-20T13:00:06.834 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-20T13:00:06.834 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":1,"fsid":"680ee126-cfee-11ee-95bb-87774f69a715","modified":"2024-02-20T12:49:44.699059Z","created":"2024-02-20T12:49:44.699059Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi064","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:3300","nonce":0},{"type":"v1","addr":"172.21.15.64:6789","nonce":0}]},"addr":"172.21.15.64:6789/0","public_addr":"172.21.15.64:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-20T13:00:06.835 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:06.834+0000 7fd280e54700 1 -- 172.21.15.71:0/1227659838 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fd268031090 msgr2=0x7fd268033550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T13:00:06.835 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:06.834+0000 7fd280e54700 1 --2- 172.21.15.71:0/1227659838 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fd268031090 0x7fd268033550 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 rx=0x7fd26c0096a0 tx=0x7fd26c006b40).stop 2024-02-20T13:00:06.835 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:06.835+0000 7fd280e54700 1 -- 172.21.15.71:0/1227659838 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd27c0f5480 msgr2=0x7fd27c1056f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T13:00:06.835 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:06.835+0000 7fd280e54700 1 --2- 172.21.15.71:0/1227659838 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd27c0f5480 0x7fd27c1056f0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 rx=0x7fd2640089f0 tx=0x7fd26400b040).stop 2024-02-20T13:00:06.835 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:06.835+0000 7fd280e54700 1 -- 172.21.15.71:0/1227659838 shutdown_connections 2024-02-20T13:00:06.835 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:06.835+0000 7fd280e54700 1 --2- 172.21.15.71:0/1227659838 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fd268031090 0x7fd268033550 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T13:00:06.835 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:06.835+0000 7fd280e54700 1 --2- 172.21.15.71:0/1227659838 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd27c0f5480 0x7fd27c1056f0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T13:00:06.836 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:06.835+0000 7fd280e54700 1 -- 172.21.15.71:0/1227659838 >> 172.21.15.71:0/1227659838 conn(0x7fd27c0f0be0 msgr2=0x7fd27c0f16d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T13:00:06.836 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:06.835+0000 7fd280e54700 1 -- 172.21.15.71:0/1227659838 shutdown_connections 2024-02-20T13:00:06.836 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:06.835+0000 7fd280e54700 1 -- 172.21.15.71:0/1227659838 wait complete. 2024-02-20T13:00:06.836 INFO:teuthology.orchestra.run.smithi071.stderr:dumped monmap epoch 1 2024-02-20T13:00:07.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 13:00:07 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T13:00:05.794202+0000 mgr.smithi064.vriyor (mgr.14156) 287 : cluster [DBG] pgmap v270: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T13:00:07.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 13:00:07 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T13:00:06.833803+0000 mon.smithi064 (mon.0) 272 : audit [DBG] from='client.? 172.21.15.71:0/1227659838' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-20T13:00:08.613 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-20T13:00:08.613 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/daemon-base:latest-pacific shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 680ee126-cfee-11ee-95bb-87774f69a715 -- ceph mon dump -f json 2024-02-20T13:00:09.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 13:00:09 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T13:00:07.794560+0000 mgr.smithi064.vriyor (mgr.14156) 288 : cluster [DBG] pgmap v271: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T13:00:10.514 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:10.501+0000 7f9a06138700 1 -- 172.21.15.71:0/3393743652 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9a000f52d0 msgr2=0x7f9a000f56f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T13:00:10.514 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:10.501+0000 7f9a06138700 1 --2- 172.21.15.71:0/3393743652 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9a000f52d0 0x7f9a000f56f0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 rx=0x7f99fc009fb0 tx=0x7f99fc009b30).stop 2024-02-20T13:00:10.515 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:10.501+0000 7f9a06138700 1 -- 172.21.15.71:0/3393743652 shutdown_connections 2024-02-20T13:00:10.515 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:10.501+0000 7f9a06138700 1 --2- 172.21.15.71:0/3393743652 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9a000f52d0 0x7f9a000f56f0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T13:00:10.515 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:10.501+0000 7f9a06138700 1 -- 172.21.15.71:0/3393743652 >> 172.21.15.71:0/3393743652 conn(0x7f9a000f0a30 msgr2=0x7f9a000f2e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T13:00:10.515 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:10.501+0000 7f9a06138700 1 -- 172.21.15.71:0/3393743652 shutdown_connections 2024-02-20T13:00:10.515 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:10.501+0000 7f9a06138700 1 -- 172.21.15.71:0/3393743652 wait complete. 2024-02-20T13:00:10.515 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:10.501+0000 7f9a06138700 1 Processor -- start 2024-02-20T13:00:10.515 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:10.501+0000 7f9a06138700 1 -- start start 2024-02-20T13:00:10.515 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:10.501+0000 7f9a06138700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9a000f52d0 0x7f9a000fd2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T13:00:10.515 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:10.502+0000 7f9a06138700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a000fb730 con 0x7f9a000f52d0 2024-02-20T13:00:10.515 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:10.502+0000 7f9a05136700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9a000f52d0 0x7f9a000fd2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T13:00:10.515 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:10.502+0000 7f9a05136700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9a000f52d0 0x7f9a000fd2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.71:39610/0 (socket says 172.21.15.71:39610) 2024-02-20T13:00:10.515 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:10.502+0000 7f9a05136700 1 -- 172.21.15.71:0/3279472900 learned_addr learned my addr 172.21.15.71:0/3279472900 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-02-20T13:00:10.515 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:10.502+0000 7f9a05136700 1 -- 172.21.15.71:0/3279472900 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99fc005040 con 0x7f9a000f52d0 2024-02-20T13:00:10.515 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:10.502+0000 7f9a05136700 1 --2- 172.21.15.71:0/3279472900 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9a000f52d0 0x7f9a000fd2b0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 rx=0x7f99fc0089f0 tx=0x7f99fc00b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T13:00:10.515 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:10.502+0000 7f99f5ffb700 1 -- 172.21.15.71:0/3279472900 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f99fc009320 con 0x7f9a000f52d0 2024-02-20T13:00:10.516 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:10.502+0000 7f99f5ffb700 1 -- 172.21.15.71:0/3279472900 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(25 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f99fc013ec0 con 0x7f9a000f52d0 2024-02-20T13:00:10.516 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:10.502+0000 7f99f5ffb700 1 -- 172.21.15.71:0/3279472900 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f99fc014d30 con 0x7f9a000f52d0 2024-02-20T13:00:10.516 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:10.502+0000 7f9a06138700 1 -- 172.21.15.71:0/3279472900 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a000fb8d0 con 0x7f9a000f52d0 2024-02-20T13:00:10.516 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:10.503+0000 7f9a06138700 1 -- 172.21.15.71:0/3279472900 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a000fbd60 con 0x7f9a000f52d0 2024-02-20T13:00:10.516 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:10.503+0000 7f99f5ffb700 1 -- 172.21.15.71:0/3279472900 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 37151+0+0 (secure 0 0 0) 0x7f99fc0095f0 con 0x7f9a000f52d0 2024-02-20T13:00:10.516 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:10.503+0000 7f99f5ffb700 1 --2- 172.21.15.71:0/3279472900 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f99ec0310e0 0x7f99ec0335a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T13:00:10.516 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:10.503+0000 7f9a06138700 1 -- 172.21.15.71:0/3279472900 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a000591e0 con 0x7f9a000f52d0 2024-02-20T13:00:10.516 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:10.504+0000 7f99f5ffb700 1 -- 172.21.15.71:0/3279472900 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1065+0+0 (secure 0 0 0) 0x7f99fc0423a0 con 0x7f9a000f52d0 2024-02-20T13:00:10.516 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:10.504+0000 7f9a04935700 1 --2- 172.21.15.71:0/3279472900 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f99ec0310e0 0x7f99ec0335a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T13:00:10.516 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:10.504+0000 7f9a04935700 1 --2- 172.21.15.71:0/3279472900 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f99ec0310e0 0x7f99ec0335a0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 rx=0x7f99f00096a0 tx=0x7f99f0006b40).ready entity=mgr.14156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T13:00:10.516 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:10.509+0000 7f99f5ffb700 1 -- 172.21.15.71:0/3279472900 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+153237 (secure 0 0 0) 0x7f99fc00fe40 con 0x7f9a000f52d0 2024-02-20T13:00:10.718 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:10.717+0000 7f9a06138700 1 -- 172.21.15.71:0/3279472900 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f9a000fadc0 con 0x7f9a000f52d0 2024-02-20T13:00:10.718 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:10.718+0000 7f99f5ffb700 1 -- 172.21.15.71:0/3279472900 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7f99fc0170c0 con 0x7f9a000f52d0 2024-02-20T13:00:10.718 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-20T13:00:10.719 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":1,"fsid":"680ee126-cfee-11ee-95bb-87774f69a715","modified":"2024-02-20T12:49:44.699059Z","created":"2024-02-20T12:49:44.699059Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi064","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:3300","nonce":0},{"type":"v1","addr":"172.21.15.64:6789","nonce":0}]},"addr":"172.21.15.64:6789/0","public_addr":"172.21.15.64:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-20T13:00:10.720 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:10.719+0000 7f9a06138700 1 -- 172.21.15.71:0/3279472900 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f99ec0310e0 msgr2=0x7f99ec0335a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T13:00:10.720 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:10.719+0000 7f9a06138700 1 --2- 172.21.15.71:0/3279472900 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f99ec0310e0 0x7f99ec0335a0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 rx=0x7f99f00096a0 tx=0x7f99f0006b40).stop 2024-02-20T13:00:10.720 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:10.719+0000 7f9a06138700 1 -- 172.21.15.71:0/3279472900 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9a000f52d0 msgr2=0x7f9a000fd2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T13:00:10.720 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:10.719+0000 7f9a06138700 1 --2- 172.21.15.71:0/3279472900 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9a000f52d0 0x7f9a000fd2b0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 rx=0x7f99fc0089f0 tx=0x7f99fc00b040).stop 2024-02-20T13:00:10.720 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:10.719+0000 7f9a06138700 1 -- 172.21.15.71:0/3279472900 shutdown_connections 2024-02-20T13:00:10.720 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:10.719+0000 7f9a06138700 1 --2- 172.21.15.71:0/3279472900 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f99ec0310e0 0x7f99ec0335a0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T13:00:10.721 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:10.719+0000 7f9a06138700 1 --2- 172.21.15.71:0/3279472900 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9a000f52d0 0x7f9a000fd2b0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T13:00:10.721 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:10.719+0000 7f9a06138700 1 -- 172.21.15.71:0/3279472900 >> 172.21.15.71:0/3279472900 conn(0x7f9a000f0a30 msgr2=0x7f9a0005fee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T13:00:10.721 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:10.719+0000 7f9a06138700 1 -- 172.21.15.71:0/3279472900 shutdown_connections 2024-02-20T13:00:10.721 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:10.719+0000 7f9a06138700 1 -- 172.21.15.71:0/3279472900 wait complete. 2024-02-20T13:00:10.721 INFO:teuthology.orchestra.run.smithi071.stderr:dumped monmap epoch 1 2024-02-20T13:00:11.563 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 13:00:11 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T13:00:09.794937+0000 mgr.smithi064.vriyor (mgr.14156) 289 : cluster [DBG] pgmap v272: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T13:00:11.563 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 13:00:11 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T13:00:10.718706+0000 mon.smithi064 (mon.0) 273 : audit [DBG] from='client.? 172.21.15.71:0/3279472900' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-20T13:00:12.548 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-20T13:00:12.548 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/daemon-base:latest-pacific shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 680ee126-cfee-11ee-95bb-87774f69a715 -- ceph mon dump -f json 2024-02-20T13:00:13.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 13:00:13 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T13:00:11.795330+0000 mgr.smithi064.vriyor (mgr.14156) 290 : cluster [DBG] pgmap v273: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T13:00:14.552 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:14.523+0000 7f62e348e700 1 -- 172.21.15.71:0/3598637359 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f62dc0f5480 msgr2=0x7f62dc0f58a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T13:00:14.553 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:14.523+0000 7f62e348e700 1 --2- 172.21.15.71:0/3598637359 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f62dc0f5480 0x7f62dc0f58a0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 rx=0x7f62cc009fb0 tx=0x7f62cc009b30).stop 2024-02-20T13:00:14.553 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:14.523+0000 7f62e348e700 1 -- 172.21.15.71:0/3598637359 shutdown_connections 2024-02-20T13:00:14.553 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:14.523+0000 7f62e348e700 1 --2- 172.21.15.71:0/3598637359 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f62dc0f5480 0x7f62dc0f58a0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T13:00:14.553 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:14.523+0000 7f62e348e700 1 -- 172.21.15.71:0/3598637359 >> 172.21.15.71:0/3598637359 conn(0x7f62dc0f0be0 msgr2=0x7f62dc0f3000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T13:00:14.553 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:14.523+0000 7f62e348e700 1 -- 172.21.15.71:0/3598637359 shutdown_connections 2024-02-20T13:00:14.553 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:14.523+0000 7f62e348e700 1 -- 172.21.15.71:0/3598637359 wait complete. 2024-02-20T13:00:14.553 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:14.524+0000 7f62e348e700 1 Processor -- start 2024-02-20T13:00:14.553 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:14.524+0000 7f62e348e700 1 -- start start 2024-02-20T13:00:14.553 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:14.524+0000 7f62e348e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f62dc0f5480 0x7f62dc109510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T13:00:14.553 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:14.524+0000 7f62e348e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62dc060300 con 0x7f62dc0f5480 2024-02-20T13:00:14.553 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:14.524+0000 7f62e122a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f62dc0f5480 0x7f62dc109510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T13:00:14.553 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:14.524+0000 7f62e122a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f62dc0f5480 0x7f62dc109510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.71:39624/0 (socket says 172.21.15.71:39624) 2024-02-20T13:00:14.554 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:14.524+0000 7f62e122a700 1 -- 172.21.15.71:0/140522483 learned_addr learned my addr 172.21.15.71:0/140522483 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-02-20T13:00:14.554 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:14.525+0000 7f62e122a700 1 -- 172.21.15.71:0/140522483 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62cc005040 con 0x7f62dc0f5480 2024-02-20T13:00:14.554 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:14.525+0000 7f62e122a700 1 --2- 172.21.15.71:0/140522483 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f62dc0f5480 0x7f62dc109510 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 rx=0x7f62cc0089f0 tx=0x7f62cc00b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T13:00:14.554 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:14.525+0000 7f62d1ffb700 1 -- 172.21.15.71:0/140522483 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f62cc009320 con 0x7f62dc0f5480 2024-02-20T13:00:14.554 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:14.525+0000 7f62e348e700 1 -- 172.21.15.71:0/140522483 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62dc060500 con 0x7f62dc0f5480 2024-02-20T13:00:14.554 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:14.525+0000 7f62d1ffb700 1 -- 172.21.15.71:0/140522483 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(25 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f62cc013ec0 con 0x7f62dc0f5480 2024-02-20T13:00:14.554 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:14.525+0000 7f62d1ffb700 1 -- 172.21.15.71:0/140522483 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f62cc014d30 con 0x7f62dc0f5480 2024-02-20T13:00:14.554 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:14.525+0000 7f62e348e700 1 -- 172.21.15.71:0/140522483 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62dc109bc0 con 0x7f62dc0f5480 2024-02-20T13:00:14.554 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:14.526+0000 7f62d1ffb700 1 -- 172.21.15.71:0/140522483 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 37151+0+0 (secure 0 0 0) 0x7f62cc0095f0 con 0x7f62dc0f5480 2024-02-20T13:00:14.554 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:14.526+0000 7f62d1ffb700 1 --2- 172.21.15.71:0/140522483 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f62c8031090 0x7f62c8033550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T13:00:14.554 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:14.526+0000 7f62e348e700 1 -- 172.21.15.71:0/140522483 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62dc0591e0 con 0x7f62dc0f5480 2024-02-20T13:00:14.554 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:14.526+0000 7f62d1ffb700 1 -- 172.21.15.71:0/140522483 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1065+0+0 (secure 0 0 0) 0x7f62cc0424a0 con 0x7f62dc0f5480 2024-02-20T13:00:14.554 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:14.526+0000 7f62e0a29700 1 --2- 172.21.15.71:0/140522483 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f62c8031090 0x7f62c8033550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T13:00:14.554 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:14.527+0000 7f62e0a29700 1 --2- 172.21.15.71:0/140522483 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f62c8031090 0x7f62c8033550 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 rx=0x7f62d80096a0 tx=0x7f62d8006b40).ready entity=mgr.14156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T13:00:14.555 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:14.531+0000 7f62d1ffb700 1 -- 172.21.15.71:0/140522483 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+153237 (secure 0 0 0) 0x7f62cc01d7f0 con 0x7f62dc0f5480 2024-02-20T13:00:14.742 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:14.740+0000 7f62e348e700 1 -- 172.21.15.71:0/140522483 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f62dc045bb0 con 0x7f62dc0f5480 2024-02-20T13:00:14.742 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:14.740+0000 7f62d1ffb700 1 -- 172.21.15.71:0/140522483 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7f62cc01d7f0 con 0x7f62dc0f5480 2024-02-20T13:00:14.742 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-20T13:00:14.742 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":1,"fsid":"680ee126-cfee-11ee-95bb-87774f69a715","modified":"2024-02-20T12:49:44.699059Z","created":"2024-02-20T12:49:44.699059Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi064","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:3300","nonce":0},{"type":"v1","addr":"172.21.15.64:6789","nonce":0}]},"addr":"172.21.15.64:6789/0","public_addr":"172.21.15.64:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-20T13:00:14.743 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:14.742+0000 7f62e348e700 1 -- 172.21.15.71:0/140522483 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f62c8031090 msgr2=0x7f62c8033550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T13:00:14.744 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:14.742+0000 7f62e348e700 1 --2- 172.21.15.71:0/140522483 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f62c8031090 0x7f62c8033550 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 rx=0x7f62d80096a0 tx=0x7f62d8006b40).stop 2024-02-20T13:00:14.744 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:14.742+0000 7f62e348e700 1 -- 172.21.15.71:0/140522483 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f62dc0f5480 msgr2=0x7f62dc109510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T13:00:14.744 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:14.742+0000 7f62e348e700 1 --2- 172.21.15.71:0/140522483 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f62dc0f5480 0x7f62dc109510 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 rx=0x7f62cc0089f0 tx=0x7f62cc00b040).stop 2024-02-20T13:00:14.744 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:14.742+0000 7f62e348e700 1 -- 172.21.15.71:0/140522483 shutdown_connections 2024-02-20T13:00:14.744 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:14.742+0000 7f62e348e700 1 --2- 172.21.15.71:0/140522483 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f62c8031090 0x7f62c8033550 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T13:00:14.744 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:14.742+0000 7f62e348e700 1 --2- 172.21.15.71:0/140522483 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f62dc0f5480 0x7f62dc109510 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T13:00:14.744 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:14.742+0000 7f62e348e700 1 -- 172.21.15.71:0/140522483 >> 172.21.15.71:0/140522483 conn(0x7f62dc0f0be0 msgr2=0x7f62dc0f16d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T13:00:14.744 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:14.742+0000 7f62e348e700 1 -- 172.21.15.71:0/140522483 shutdown_connections 2024-02-20T13:00:14.744 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:14.742+0000 7f62e348e700 1 -- 172.21.15.71:0/140522483 wait complete. 2024-02-20T13:00:14.745 INFO:teuthology.orchestra.run.smithi071.stderr:dumped monmap epoch 1 2024-02-20T13:00:15.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 13:00:15 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T13:00:13.795667+0000 mgr.smithi064.vriyor (mgr.14156) 291 : cluster [DBG] pgmap v274: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T13:00:15.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 13:00:15 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T13:00:14.742095+0000 mon.smithi064 (mon.0) 274 : audit [DBG] from='client.? 172.21.15.71:0/140522483' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-20T13:00:16.449 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-20T13:00:16.449 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/daemon-base:latest-pacific shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 680ee126-cfee-11ee-95bb-87774f69a715 -- ceph mon dump -f json 2024-02-20T13:00:17.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 13:00:17 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T13:00:15.796028+0000 mgr.smithi064.vriyor (mgr.14156) 292 : cluster [DBG] pgmap v275: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T13:00:18.423 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:18.420+0000 7fc35863e700 1 -- 172.21.15.71:0/3687132425 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc3500f4850 msgr2=0x7fc3500f4c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T13:00:18.423 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:18.420+0000 7fc35863e700 1 --2- 172.21.15.71:0/3687132425 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc3500f4850 0x7fc3500f4c70 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 rx=0x7fc340009fb0 tx=0x7fc340009b30).stop 2024-02-20T13:00:18.424 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:18.420+0000 7fc35863e700 1 -- 172.21.15.71:0/3687132425 shutdown_connections 2024-02-20T13:00:18.424 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:18.420+0000 7fc35863e700 1 --2- 172.21.15.71:0/3687132425 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc3500f4850 0x7fc3500f4c70 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T13:00:18.424 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:18.420+0000 7fc35863e700 1 -- 172.21.15.71:0/3687132425 >> 172.21.15.71:0/3687132425 conn(0x7fc3500f0390 msgr2=0x7fc3500f27b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T13:00:18.424 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:18.420+0000 7fc35863e700 1 -- 172.21.15.71:0/3687132425 shutdown_connections 2024-02-20T13:00:18.424 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:18.420+0000 7fc35863e700 1 -- 172.21.15.71:0/3687132425 wait complete. 2024-02-20T13:00:18.424 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:18.421+0000 7fc35863e700 1 Processor -- start 2024-02-20T13:00:18.424 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:18.421+0000 7fc35863e700 1 -- start start 2024-02-20T13:00:18.424 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:18.421+0000 7fc35863e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc3500f4850 0x7fc350106780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T13:00:18.424 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:18.421+0000 7fc35863e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc350106c70 con 0x7fc3500f4850 2024-02-20T13:00:18.424 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:18.421+0000 7fc3563da700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc3500f4850 0x7fc350106780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T13:00:18.424 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:18.421+0000 7fc3563da700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc3500f4850 0x7fc350106780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.71:53720/0 (socket says 172.21.15.71:53720) 2024-02-20T13:00:18.424 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:18.421+0000 7fc3563da700 1 -- 172.21.15.71:0/1457372462 learned_addr learned my addr 172.21.15.71:0/1457372462 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-02-20T13:00:18.424 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:18.421+0000 7fc3563da700 1 -- 172.21.15.71:0/1457372462 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc340005040 con 0x7fc3500f4850 2024-02-20T13:00:18.425 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:18.421+0000 7fc3563da700 1 --2- 172.21.15.71:0/1457372462 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc3500f4850 0x7fc350106780 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 rx=0x7fc3400089f0 tx=0x7fc34000b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T13:00:18.425 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:18.422+0000 7fc346ffd700 1 -- 172.21.15.71:0/1457372462 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc340009320 con 0x7fc3500f4850 2024-02-20T13:00:18.425 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:18.422+0000 7fc346ffd700 1 -- 172.21.15.71:0/1457372462 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(25 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7fc340013ec0 con 0x7fc3500f4850 2024-02-20T13:00:18.425 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:18.422+0000 7fc346ffd700 1 -- 172.21.15.71:0/1457372462 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc340014d30 con 0x7fc3500f4850 2024-02-20T13:00:18.425 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:18.422+0000 7fc35863e700 1 -- 172.21.15.71:0/1457372462 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc350106e70 con 0x7fc3500f4850 2024-02-20T13:00:18.425 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:18.422+0000 7fc35863e700 1 -- 172.21.15.71:0/1457372462 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc350107300 con 0x7fc3500f4850 2024-02-20T13:00:18.426 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:18.423+0000 7fc346ffd700 1 -- 172.21.15.71:0/1457372462 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 37151+0+0 (secure 0 0 0) 0x7fc340014420 con 0x7fc3500f4850 2024-02-20T13:00:18.426 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:18.423+0000 7fc346ffd700 1 --2- 172.21.15.71:0/1457372462 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fc33c0310e0 0x7fc33c0335a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T13:00:18.426 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:18.423+0000 7fc346ffd700 1 -- 172.21.15.71:0/1457372462 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1065+0+0 (secure 0 0 0) 0x7fc340042420 con 0x7fc3500f4850 2024-02-20T13:00:18.426 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:18.423+0000 7fc35863e700 1 -- 172.21.15.71:0/1457372462 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc350045bb0 con 0x7fc3500f4850 2024-02-20T13:00:18.426 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:18.423+0000 7fc355bd9700 1 --2- 172.21.15.71:0/1457372462 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fc33c0310e0 0x7fc33c0335a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T13:00:18.426 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:18.423+0000 7fc355bd9700 1 --2- 172.21.15.71:0/1457372462 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fc33c0310e0 0x7fc33c0335a0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 rx=0x7fc34c0096a0 tx=0x7fc34c006b40).ready entity=mgr.14156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T13:00:18.429 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:18.428+0000 7fc346ffd700 1 -- 172.21.15.71:0/1457372462 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+153237 (secure 0 0 0) 0x7fc340014750 con 0x7fc3500f4850 2024-02-20T13:00:18.635 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:18.634+0000 7fc35863e700 1 -- 172.21.15.71:0/1457372462 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fc350107fb0 con 0x7fc3500f4850 2024-02-20T13:00:18.636 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:18.635+0000 7fc346ffd700 1 -- 172.21.15.71:0/1457372462 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7fc34001d7f0 con 0x7fc3500f4850 2024-02-20T13:00:18.636 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-20T13:00:18.636 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":1,"fsid":"680ee126-cfee-11ee-95bb-87774f69a715","modified":"2024-02-20T12:49:44.699059Z","created":"2024-02-20T12:49:44.699059Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi064","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:3300","nonce":0},{"type":"v1","addr":"172.21.15.64:6789","nonce":0}]},"addr":"172.21.15.64:6789/0","public_addr":"172.21.15.64:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-20T13:00:18.638 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:18.636+0000 7fc35863e700 1 -- 172.21.15.71:0/1457372462 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fc33c0310e0 msgr2=0x7fc33c0335a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T13:00:18.638 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:18.636+0000 7fc35863e700 1 --2- 172.21.15.71:0/1457372462 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fc33c0310e0 0x7fc33c0335a0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 rx=0x7fc34c0096a0 tx=0x7fc34c006b40).stop 2024-02-20T13:00:18.638 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:18.636+0000 7fc35863e700 1 -- 172.21.15.71:0/1457372462 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc3500f4850 msgr2=0x7fc350106780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T13:00:18.638 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:18.636+0000 7fc35863e700 1 --2- 172.21.15.71:0/1457372462 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc3500f4850 0x7fc350106780 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 rx=0x7fc3400089f0 tx=0x7fc34000b040).stop 2024-02-20T13:00:18.638 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:18.636+0000 7fc35863e700 1 -- 172.21.15.71:0/1457372462 shutdown_connections 2024-02-20T13:00:18.638 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:18.636+0000 7fc35863e700 1 --2- 172.21.15.71:0/1457372462 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fc33c0310e0 0x7fc33c0335a0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T13:00:18.638 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:18.636+0000 7fc35863e700 1 --2- 172.21.15.71:0/1457372462 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc3500f4850 0x7fc350106780 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T13:00:18.638 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:18.636+0000 7fc35863e700 1 -- 172.21.15.71:0/1457372462 >> 172.21.15.71:0/1457372462 conn(0x7fc3500f0390 msgr2=0x7fc3500f0e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T13:00:18.638 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:18.636+0000 7fc35863e700 1 -- 172.21.15.71:0/1457372462 shutdown_connections 2024-02-20T13:00:18.638 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:18.636+0000 7fc35863e700 1 -- 172.21.15.71:0/1457372462 wait complete. 2024-02-20T13:00:18.639 INFO:teuthology.orchestra.run.smithi071.stderr:dumped monmap epoch 1 2024-02-20T13:00:19.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 13:00:19 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T13:00:17.796416+0000 mgr.smithi064.vriyor (mgr.14156) 293 : cluster [DBG] pgmap v276: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T13:00:19.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 13:00:19 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T13:00:18.636255+0000 mon.smithi064 (mon.0) 275 : audit [DBG] from='client.? 172.21.15.71:0/1457372462' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-20T13:00:20.409 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-20T13:00:20.410 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/daemon-base:latest-pacific shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 680ee126-cfee-11ee-95bb-87774f69a715 -- ceph mon dump -f json 2024-02-20T13:00:21.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 13:00:21 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T13:00:19.796793+0000 mgr.smithi064.vriyor (mgr.14156) 294 : cluster [DBG] pgmap v277: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T13:00:22.265 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:22.209+0000 7f689927d700 1 -- 172.21.15.71:0/2067372124 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f68940f54b0 msgr2=0x7f68940f58d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T13:00:22.265 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:22.209+0000 7f689927d700 1 --2- 172.21.15.71:0/2067372124 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f68940f54b0 0x7f68940f58d0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 rx=0x7f687c009fb0 tx=0x7f687c009b30).stop 2024-02-20T13:00:22.265 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:22.209+0000 7f689927d700 1 -- 172.21.15.71:0/2067372124 shutdown_connections 2024-02-20T13:00:22.265 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:22.209+0000 7f689927d700 1 --2- 172.21.15.71:0/2067372124 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f68940f54b0 0x7f68940f58d0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T13:00:22.265 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:22.209+0000 7f689927d700 1 -- 172.21.15.71:0/2067372124 >> 172.21.15.71:0/2067372124 conn(0x7f68940f0c10 msgr2=0x7f68940f3030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T13:00:22.265 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:22.209+0000 7f689927d700 1 -- 172.21.15.71:0/2067372124 shutdown_connections 2024-02-20T13:00:22.265 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:22.209+0000 7f689927d700 1 -- 172.21.15.71:0/2067372124 wait complete. 2024-02-20T13:00:22.265 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:22.210+0000 7f689927d700 1 Processor -- start 2024-02-20T13:00:22.265 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:22.210+0000 7f689927d700 1 -- start start 2024-02-20T13:00:22.265 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:22.210+0000 7f689927d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f68940f54b0 0x7f68941094f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T13:00:22.266 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:22.210+0000 7f689927d700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6894060320 con 0x7f68940f54b0 2024-02-20T13:00:22.266 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:22.210+0000 7f6892ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f68940f54b0 0x7f68941094f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T13:00:22.266 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:22.210+0000 7f6892ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f68940f54b0 0x7f68941094f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.71:53736/0 (socket says 172.21.15.71:53736) 2024-02-20T13:00:22.266 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:22.210+0000 7f6892ffd700 1 -- 172.21.15.71:0/2831281642 learned_addr learned my addr 172.21.15.71:0/2831281642 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-02-20T13:00:22.266 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:22.211+0000 7f6892ffd700 1 -- 172.21.15.71:0/2831281642 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f687c005040 con 0x7f68940f54b0 2024-02-20T13:00:22.266 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:22.211+0000 7f6892ffd700 1 --2- 172.21.15.71:0/2831281642 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f68940f54b0 0x7f68941094f0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 rx=0x7f687c000c00 tx=0x7f687c00b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T13:00:22.266 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:22.211+0000 7f688b7fe700 1 -- 172.21.15.71:0/2831281642 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f687c0095f0 con 0x7f68940f54b0 2024-02-20T13:00:22.266 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:22.211+0000 7f689927d700 1 -- 172.21.15.71:0/2831281642 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6894060520 con 0x7f68940f54b0 2024-02-20T13:00:22.266 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:22.211+0000 7f688b7fe700 1 -- 172.21.15.71:0/2831281642 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(25 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f687c007e50 con 0x7f68940f54b0 2024-02-20T13:00:22.266 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:22.211+0000 7f688b7fe700 1 -- 172.21.15.71:0/2831281642 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f687c014e70 con 0x7f68940f54b0 2024-02-20T13:00:22.266 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:22.211+0000 7f689927d700 1 -- 172.21.15.71:0/2831281642 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6894109ba0 con 0x7f68940f54b0 2024-02-20T13:00:22.266 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:22.212+0000 7f688b7fe700 1 -- 172.21.15.71:0/2831281642 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 37151+0+0 (secure 0 0 0) 0x7f687c025070 con 0x7f68940f54b0 2024-02-20T13:00:22.266 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:22.212+0000 7f688b7fe700 1 --2- 172.21.15.71:0/2831281642 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f68800310e0 0x7f68800335a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T13:00:22.266 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:22.212+0000 7f689927d700 1 -- 172.21.15.71:0/2831281642 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6894045bb0 con 0x7f68940f54b0 2024-02-20T13:00:22.266 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:22.212+0000 7f688b7fe700 1 -- 172.21.15.71:0/2831281642 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1065+0+0 (secure 0 0 0) 0x7f687c041de0 con 0x7f68940f54b0 2024-02-20T13:00:22.267 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:22.212+0000 7f68927fc700 1 --2- 172.21.15.71:0/2831281642 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f68800310e0 0x7f68800335a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T13:00:22.267 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:22.213+0000 7f68927fc700 1 --2- 172.21.15.71:0/2831281642 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f68800310e0 0x7f68800335a0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 rx=0x7f68840096a0 tx=0x7f6884006b40).ready entity=mgr.14156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T13:00:22.267 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:22.217+0000 7f688b7fe700 1 -- 172.21.15.71:0/2831281642 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+153237 (secure 0 0 0) 0x7f687c0079c0 con 0x7f68940f54b0 2024-02-20T13:00:22.424 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:22.422+0000 7f689927d700 1 -- 172.21.15.71:0/2831281642 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f68940fb090 con 0x7f68940f54b0 2024-02-20T13:00:22.425 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:22.423+0000 7f688b7fe700 1 -- 172.21.15.71:0/2831281642 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7f687c02d090 con 0x7f68940f54b0 2024-02-20T13:00:22.425 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-20T13:00:22.425 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":1,"fsid":"680ee126-cfee-11ee-95bb-87774f69a715","modified":"2024-02-20T12:49:44.699059Z","created":"2024-02-20T12:49:44.699059Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi064","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:3300","nonce":0},{"type":"v1","addr":"172.21.15.64:6789","nonce":0}]},"addr":"172.21.15.64:6789/0","public_addr":"172.21.15.64:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-20T13:00:22.426 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:22.425+0000 7f689927d700 1 -- 172.21.15.71:0/2831281642 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f68800310e0 msgr2=0x7f68800335a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T13:00:22.426 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:22.425+0000 7f689927d700 1 --2- 172.21.15.71:0/2831281642 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f68800310e0 0x7f68800335a0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 rx=0x7f68840096a0 tx=0x7f6884006b40).stop 2024-02-20T13:00:22.426 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:22.425+0000 7f689927d700 1 -- 172.21.15.71:0/2831281642 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f68940f54b0 msgr2=0x7f68941094f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T13:00:22.427 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:22.425+0000 7f689927d700 1 --2- 172.21.15.71:0/2831281642 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f68940f54b0 0x7f68941094f0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 rx=0x7f687c000c00 tx=0x7f687c00b040).stop 2024-02-20T13:00:22.427 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:22.425+0000 7f689927d700 1 -- 172.21.15.71:0/2831281642 shutdown_connections 2024-02-20T13:00:22.427 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:22.425+0000 7f689927d700 1 --2- 172.21.15.71:0/2831281642 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f68800310e0 0x7f68800335a0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T13:00:22.427 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:22.425+0000 7f689927d700 1 --2- 172.21.15.71:0/2831281642 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f68940f54b0 0x7f68941094f0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T13:00:22.427 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:22.425+0000 7f689927d700 1 -- 172.21.15.71:0/2831281642 >> 172.21.15.71:0/2831281642 conn(0x7f68940f0c10 msgr2=0x7f68940f16d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T13:00:22.427 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:22.425+0000 7f689927d700 1 -- 172.21.15.71:0/2831281642 shutdown_connections 2024-02-20T13:00:22.427 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:22.425+0000 7f689927d700 1 -- 172.21.15.71:0/2831281642 wait complete. 2024-02-20T13:00:22.427 INFO:teuthology.orchestra.run.smithi071.stderr:dumped monmap epoch 1 2024-02-20T13:00:23.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 13:00:23 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T13:00:21.797171+0000 mgr.smithi064.vriyor (mgr.14156) 295 : cluster [DBG] pgmap v278: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T13:00:23.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 13:00:23 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T13:00:22.425142+0000 mon.smithi064 (mon.0) 276 : audit [DBG] from='client.? 172.21.15.71:0/2831281642' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-20T13:00:24.198 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-20T13:00:24.198 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/daemon-base:latest-pacific shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 680ee126-cfee-11ee-95bb-87774f69a715 -- ceph mon dump -f json 2024-02-20T13:00:25.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 13:00:25 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T13:00:23.797443+0000 mgr.smithi064.vriyor (mgr.14156) 296 : cluster [DBG] pgmap v279: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T13:00:26.254 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:26.234+0000 7f0ca7ca3700 1 -- 172.21.15.71:0/2021893904 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0ca00f5480 msgr2=0x7f0ca00f58a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T13:00:26.254 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:26.234+0000 7f0ca7ca3700 1 --2- 172.21.15.71:0/2021893904 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0ca00f5480 0x7f0ca00f58a0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 rx=0x7f0c94009fb0 tx=0x7f0c94009b30).stop 2024-02-20T13:00:26.254 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:26.234+0000 7f0ca7ca3700 1 -- 172.21.15.71:0/2021893904 shutdown_connections 2024-02-20T13:00:26.254 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:26.234+0000 7f0ca7ca3700 1 --2- 172.21.15.71:0/2021893904 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0ca00f5480 0x7f0ca00f58a0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T13:00:26.255 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:26.234+0000 7f0ca7ca3700 1 -- 172.21.15.71:0/2021893904 >> 172.21.15.71:0/2021893904 conn(0x7f0ca00f0be0 msgr2=0x7f0ca00f3000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T13:00:26.255 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:26.234+0000 7f0ca7ca3700 1 -- 172.21.15.71:0/2021893904 shutdown_connections 2024-02-20T13:00:26.255 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:26.234+0000 7f0ca7ca3700 1 -- 172.21.15.71:0/2021893904 wait complete. 2024-02-20T13:00:26.255 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:26.235+0000 7f0ca7ca3700 1 Processor -- start 2024-02-20T13:00:26.255 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:26.235+0000 7f0ca7ca3700 1 -- start start 2024-02-20T13:00:26.255 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:26.235+0000 7f0ca7ca3700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0ca00f5480 0x7f0ca01056f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T13:00:26.255 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:26.235+0000 7f0ca7ca3700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ca0105be0 con 0x7f0ca00f5480 2024-02-20T13:00:26.255 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:26.235+0000 7f0ca5a3f700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0ca00f5480 0x7f0ca01056f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T13:00:26.255 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:26.235+0000 7f0ca5a3f700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0ca00f5480 0x7f0ca01056f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.71:53760/0 (socket says 172.21.15.71:53760) 2024-02-20T13:00:26.255 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:26.235+0000 7f0ca5a3f700 1 -- 172.21.15.71:0/3241149007 learned_addr learned my addr 172.21.15.71:0/3241149007 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-02-20T13:00:26.255 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:26.235+0000 7f0ca5a3f700 1 -- 172.21.15.71:0/3241149007 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c94005040 con 0x7f0ca00f5480 2024-02-20T13:00:26.255 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:26.235+0000 7f0ca5a3f700 1 --2- 172.21.15.71:0/3241149007 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0ca00f5480 0x7f0ca01056f0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 rx=0x7f0c940089f0 tx=0x7f0c9400b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T13:00:26.255 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:26.236+0000 7f0c927fc700 1 -- 172.21.15.71:0/3241149007 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0c94009320 con 0x7f0ca00f5480 2024-02-20T13:00:26.256 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:26.236+0000 7f0ca7ca3700 1 -- 172.21.15.71:0/3241149007 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ca0105de0 con 0x7f0ca00f5480 2024-02-20T13:00:26.256 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:26.236+0000 7f0c927fc700 1 -- 172.21.15.71:0/3241149007 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(25 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f0c94013ec0 con 0x7f0ca00f5480 2024-02-20T13:00:26.256 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:26.236+0000 7f0c927fc700 1 -- 172.21.15.71:0/3241149007 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0c94014d30 con 0x7f0ca00f5480 2024-02-20T13:00:26.256 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:26.236+0000 7f0ca7ca3700 1 -- 172.21.15.71:0/3241149007 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ca0106270 con 0x7f0ca00f5480 2024-02-20T13:00:26.256 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:26.236+0000 7f0c927fc700 1 -- 172.21.15.71:0/3241149007 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 37151+0+0 (secure 0 0 0) 0x7f0c940095f0 con 0x7f0ca00f5480 2024-02-20T13:00:26.256 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:26.236+0000 7f0ca7ca3700 1 -- 172.21.15.71:0/3241149007 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ca0045bb0 con 0x7f0ca00f5480 2024-02-20T13:00:26.256 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:26.237+0000 7f0c927fc700 1 --2- 172.21.15.71:0/3241149007 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f0c8c031090 0x7f0c8c033550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T13:00:26.256 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:26.237+0000 7f0c927fc700 1 -- 172.21.15.71:0/3241149007 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1065+0+0 (secure 0 0 0) 0x7f0c940424e0 con 0x7f0ca00f5480 2024-02-20T13:00:26.256 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:26.237+0000 7f0ca523e700 1 --2- 172.21.15.71:0/3241149007 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f0c8c031090 0x7f0c8c033550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T13:00:26.256 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:26.237+0000 7f0ca523e700 1 --2- 172.21.15.71:0/3241149007 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f0c8c031090 0x7f0c8c033550 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 rx=0x7f0c9c0096a0 tx=0x7f0c9c006b40).ready entity=mgr.14156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T13:00:26.256 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:26.241+0000 7f0c927fc700 1 -- 172.21.15.71:0/3241149007 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+153237 (secure 0 0 0) 0x7f0c9401d7f0 con 0x7f0ca00f5480 2024-02-20T13:00:26.449 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:26.448+0000 7f0ca7ca3700 1 -- 172.21.15.71:0/3241149007 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f0ca00faf70 con 0x7f0ca00f5480 2024-02-20T13:00:26.450 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:26.449+0000 7f0c927fc700 1 -- 172.21.15.71:0/3241149007 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7f0c9400fe40 con 0x7f0ca00f5480 2024-02-20T13:00:26.450 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-20T13:00:26.450 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":1,"fsid":"680ee126-cfee-11ee-95bb-87774f69a715","modified":"2024-02-20T12:49:44.699059Z","created":"2024-02-20T12:49:44.699059Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi064","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:3300","nonce":0},{"type":"v1","addr":"172.21.15.64:6789","nonce":0}]},"addr":"172.21.15.64:6789/0","public_addr":"172.21.15.64:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-20T13:00:26.452 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:26.450+0000 7f0ca7ca3700 1 -- 172.21.15.71:0/3241149007 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f0c8c031090 msgr2=0x7f0c8c033550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T13:00:26.452 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:26.450+0000 7f0ca7ca3700 1 --2- 172.21.15.71:0/3241149007 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f0c8c031090 0x7f0c8c033550 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 rx=0x7f0c9c0096a0 tx=0x7f0c9c006b40).stop 2024-02-20T13:00:26.452 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:26.450+0000 7f0ca7ca3700 1 -- 172.21.15.71:0/3241149007 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0ca00f5480 msgr2=0x7f0ca01056f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T13:00:26.452 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:26.450+0000 7f0ca7ca3700 1 --2- 172.21.15.71:0/3241149007 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0ca00f5480 0x7f0ca01056f0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 rx=0x7f0c940089f0 tx=0x7f0c9400b040).stop 2024-02-20T13:00:26.452 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:26.450+0000 7f0ca7ca3700 1 -- 172.21.15.71:0/3241149007 shutdown_connections 2024-02-20T13:00:26.452 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:26.450+0000 7f0ca7ca3700 1 --2- 172.21.15.71:0/3241149007 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f0c8c031090 0x7f0c8c033550 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T13:00:26.453 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:26.450+0000 7f0ca7ca3700 1 --2- 172.21.15.71:0/3241149007 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0ca00f5480 0x7f0ca01056f0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T13:00:26.453 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:26.450+0000 7f0ca7ca3700 1 -- 172.21.15.71:0/3241149007 >> 172.21.15.71:0/3241149007 conn(0x7f0ca00f0be0 msgr2=0x7f0ca00f16d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T13:00:26.453 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:26.450+0000 7f0ca7ca3700 1 -- 172.21.15.71:0/3241149007 shutdown_connections 2024-02-20T13:00:26.453 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:26.450+0000 7f0ca7ca3700 1 -- 172.21.15.71:0/3241149007 wait complete. 2024-02-20T13:00:26.453 INFO:teuthology.orchestra.run.smithi071.stderr:dumped monmap epoch 1 2024-02-20T13:00:27.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 13:00:27 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T13:00:25.797811+0000 mgr.smithi064.vriyor (mgr.14156) 297 : cluster [DBG] pgmap v280: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T13:00:27.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 13:00:27 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T13:00:26.450447+0000 mon.smithi064 (mon.0) 277 : audit [DBG] from='client.? 172.21.15.71:0/3241149007' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-20T13:00:28.219 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-20T13:00:28.220 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/daemon-base:latest-pacific shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 680ee126-cfee-11ee-95bb-87774f69a715 -- ceph mon dump -f json 2024-02-20T13:00:29.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 13:00:29 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T13:00:27.798119+0000 mgr.smithi064.vriyor (mgr.14156) 298 : cluster [DBG] pgmap v281: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T13:00:30.175 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:30.172+0000 7f45c59b8700 1 -- 172.21.15.71:0/2857895012 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f45c00602c0 msgr2=0x7f45c00f9050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T13:00:30.175 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:30.172+0000 7f45c59b8700 1 --2- 172.21.15.71:0/2857895012 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f45c00602c0 0x7f45c00f9050 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 rx=0x7f45b4009fb0 tx=0x7f45b4009b30).stop 2024-02-20T13:00:30.175 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:30.172+0000 7f45c59b8700 1 -- 172.21.15.71:0/2857895012 shutdown_connections 2024-02-20T13:00:30.176 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:30.172+0000 7f45c59b8700 1 --2- 172.21.15.71:0/2857895012 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f45c00602c0 0x7f45c00f9050 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T13:00:30.176 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:30.172+0000 7f45c59b8700 1 -- 172.21.15.71:0/2857895012 >> 172.21.15.71:0/2857895012 conn(0x7f45c00f0a30 msgr2=0x7f45c00f2e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T13:00:30.176 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:30.172+0000 7f45c59b8700 1 -- 172.21.15.71:0/2857895012 shutdown_connections 2024-02-20T13:00:30.176 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:30.172+0000 7f45c59b8700 1 -- 172.21.15.71:0/2857895012 wait complete. 2024-02-20T13:00:30.176 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:30.172+0000 7f45c59b8700 1 Processor -- start 2024-02-20T13:00:30.176 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:30.172+0000 7f45c59b8700 1 -- start start 2024-02-20T13:00:30.176 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:30.173+0000 7f45c59b8700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f45c00602c0 0x7f45c01065b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T13:00:30.176 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:30.173+0000 7f45c59b8700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45c0106aa0 con 0x7f45c00602c0 2024-02-20T13:00:30.176 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:30.173+0000 7f45c49b6700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f45c00602c0 0x7f45c01065b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T13:00:30.176 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:30.173+0000 7f45c49b6700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f45c00602c0 0x7f45c01065b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.71:43882/0 (socket says 172.21.15.71:43882) 2024-02-20T13:00:30.176 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:30.173+0000 7f45c49b6700 1 -- 172.21.15.71:0/3786795025 learned_addr learned my addr 172.21.15.71:0/3786795025 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-02-20T13:00:30.176 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:30.173+0000 7f45c49b6700 1 -- 172.21.15.71:0/3786795025 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45b4005040 con 0x7f45c00602c0 2024-02-20T13:00:30.176 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:30.173+0000 7f45c49b6700 1 --2- 172.21.15.71:0/3786795025 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f45c00602c0 0x7f45c01065b0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 rx=0x7f45b4009d00 tx=0x7f45b4009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T13:00:30.177 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:30.174+0000 7f45bd7fa700 1 -- 172.21.15.71:0/3786795025 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f45b4018070 con 0x7f45c00602c0 2024-02-20T13:00:30.177 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:30.174+0000 7f45c59b8700 1 -- 172.21.15.71:0/3786795025 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45c0106ca0 con 0x7f45c00602c0 2024-02-20T13:00:30.177 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:30.174+0000 7f45bd7fa700 1 -- 172.21.15.71:0/3786795025 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(25 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f45b401d420 con 0x7f45c00602c0 2024-02-20T13:00:30.177 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:30.174+0000 7f45bd7fa700 1 -- 172.21.15.71:0/3786795025 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f45b401c5a0 con 0x7f45c00602c0 2024-02-20T13:00:30.177 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:30.174+0000 7f45c59b8700 1 -- 172.21.15.71:0/3786795025 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45c0107130 con 0x7f45c00602c0 2024-02-20T13:00:30.178 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:30.174+0000 7f45bd7fa700 1 -- 172.21.15.71:0/3786795025 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 37151+0+0 (secure 0 0 0) 0x7f45b401c860 con 0x7f45c00602c0 2024-02-20T13:00:30.178 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:30.175+0000 7f45bd7fa700 1 --2- 172.21.15.71:0/3786795025 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f45b0031090 0x7f45b0033550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T13:00:30.178 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:30.175+0000 7f45bd7fa700 1 -- 172.21.15.71:0/3786795025 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1065+0+0 (secure 0 0 0) 0x7f45b4042950 con 0x7f45c00602c0 2024-02-20T13:00:30.178 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:30.175+0000 7f45c59b8700 1 -- 172.21.15.71:0/3786795025 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45c00591e0 con 0x7f45c00602c0 2024-02-20T13:00:30.178 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:30.175+0000 7f45bffff700 1 --2- 172.21.15.71:0/3786795025 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f45b0031090 0x7f45b0033550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T13:00:30.178 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:30.175+0000 7f45bffff700 1 --2- 172.21.15.71:0/3786795025 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f45b0031090 0x7f45b0033550 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 rx=0x7f45ac0096a0 tx=0x7f45ac006b40).ready entity=mgr.14156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T13:00:30.181 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:30.180+0000 7f45bd7fa700 1 -- 172.21.15.71:0/3786795025 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+153237 (secure 0 0 0) 0x7f45b401bdb0 con 0x7f45c00602c0 2024-02-20T13:00:30.387 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:30.385+0000 7f45c59b8700 1 -- 172.21.15.71:0/3786795025 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f45c0045bb0 con 0x7f45c00602c0 2024-02-20T13:00:30.387 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:30.386+0000 7f45bd7fa700 1 -- 172.21.15.71:0/3786795025 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7f45b401b370 con 0x7f45c00602c0 2024-02-20T13:00:30.387 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-20T13:00:30.388 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":1,"fsid":"680ee126-cfee-11ee-95bb-87774f69a715","modified":"2024-02-20T12:49:44.699059Z","created":"2024-02-20T12:49:44.699059Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi064","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:3300","nonce":0},{"type":"v1","addr":"172.21.15.64:6789","nonce":0}]},"addr":"172.21.15.64:6789/0","public_addr":"172.21.15.64:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-20T13:00:30.389 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:30.387+0000 7f45c59b8700 1 -- 172.21.15.71:0/3786795025 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f45b0031090 msgr2=0x7f45b0033550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T13:00:30.389 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:30.387+0000 7f45c59b8700 1 --2- 172.21.15.71:0/3786795025 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f45b0031090 0x7f45b0033550 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 rx=0x7f45ac0096a0 tx=0x7f45ac006b40).stop 2024-02-20T13:00:30.389 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:30.387+0000 7f45c59b8700 1 -- 172.21.15.71:0/3786795025 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f45c00602c0 msgr2=0x7f45c01065b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T13:00:30.389 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:30.387+0000 7f45c59b8700 1 --2- 172.21.15.71:0/3786795025 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f45c00602c0 0x7f45c01065b0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 rx=0x7f45b4009d00 tx=0x7f45b4009b30).stop 2024-02-20T13:00:30.390 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:30.388+0000 7f45c59b8700 1 -- 172.21.15.71:0/3786795025 shutdown_connections 2024-02-20T13:00:30.390 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:30.388+0000 7f45c59b8700 1 --2- 172.21.15.71:0/3786795025 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f45b0031090 0x7f45b0033550 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T13:00:30.390 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:30.388+0000 7f45c59b8700 1 --2- 172.21.15.71:0/3786795025 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f45c00602c0 0x7f45c01065b0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T13:00:30.390 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:30.388+0000 7f45c59b8700 1 -- 172.21.15.71:0/3786795025 >> 172.21.15.71:0/3786795025 conn(0x7f45c00f0a30 msgr2=0x7f45c00f1520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T13:00:30.390 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:30.388+0000 7f45c59b8700 1 -- 172.21.15.71:0/3786795025 shutdown_connections 2024-02-20T13:00:30.390 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:30.388+0000 7f45c59b8700 1 -- 172.21.15.71:0/3786795025 wait complete. 2024-02-20T13:00:30.390 INFO:teuthology.orchestra.run.smithi071.stderr:dumped monmap epoch 1 2024-02-20T13:00:31.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 13:00:31 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T13:00:29.798390+0000 mgr.smithi064.vriyor (mgr.14156) 299 : cluster [DBG] pgmap v282: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T13:00:31.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 13:00:31 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T13:00:30.387812+0000 mon.smithi064 (mon.0) 278 : audit [DBG] from='client.? 172.21.15.71:0/3786795025' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-20T13:00:32.028 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-20T13:00:32.028 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/daemon-base:latest-pacific shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 680ee126-cfee-11ee-95bb-87774f69a715 -- ceph mon dump -f json 2024-02-20T13:00:33.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 13:00:33 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T13:00:31.798655+0000 mgr.smithi064.vriyor (mgr.14156) 300 : cluster [DBG] pgmap v283: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T13:00:34.162 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:34.159+0000 7f6bf6fb6700 1 -- 172.21.15.71:0/1385662880 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6bf00f63e0 msgr2=0x7f6bf00f6800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T13:00:34.163 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:34.159+0000 7f6bf6fb6700 1 --2- 172.21.15.71:0/1385662880 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6bf00f63e0 0x7f6bf00f6800 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 rx=0x7f6be0009fb0 tx=0x7f6be0009b30).stop 2024-02-20T13:00:34.163 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:34.159+0000 7f6bf6fb6700 1 -- 172.21.15.71:0/1385662880 shutdown_connections 2024-02-20T13:00:34.163 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:34.159+0000 7f6bf6fb6700 1 --2- 172.21.15.71:0/1385662880 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6bf00f63e0 0x7f6bf00f6800 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T13:00:34.163 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:34.159+0000 7f6bf6fb6700 1 -- 172.21.15.71:0/1385662880 >> 172.21.15.71:0/1385662880 conn(0x7f6bf00f1b40 msgr2=0x7f6bf00f3f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T13:00:34.163 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:34.159+0000 7f6bf6fb6700 1 -- 172.21.15.71:0/1385662880 shutdown_connections 2024-02-20T13:00:34.163 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:34.159+0000 7f6bf6fb6700 1 -- 172.21.15.71:0/1385662880 wait complete. 2024-02-20T13:00:34.163 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:34.160+0000 7f6bf6fb6700 1 Processor -- start 2024-02-20T13:00:34.163 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:34.160+0000 7f6bf6fb6700 1 -- start start 2024-02-20T13:00:34.163 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:34.160+0000 7f6bf6fb6700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6bf00f63e0 0x7f6bf0106620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T13:00:34.163 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:34.160+0000 7f6bf6fb6700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bf0106b10 con 0x7f6bf00f63e0 2024-02-20T13:00:34.163 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:34.160+0000 7f6bf5fb4700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6bf00f63e0 0x7f6bf0106620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T13:00:34.163 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:34.160+0000 7f6bf5fb4700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6bf00f63e0 0x7f6bf0106620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.71:43904/0 (socket says 172.21.15.71:43904) 2024-02-20T13:00:34.164 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:34.160+0000 7f6bf5fb4700 1 -- 172.21.15.71:0/2877723531 learned_addr learned my addr 172.21.15.71:0/2877723531 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-02-20T13:00:34.164 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:34.160+0000 7f6bf5fb4700 1 -- 172.21.15.71:0/2877723531 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6be0005040 con 0x7f6bf00f63e0 2024-02-20T13:00:34.164 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:34.161+0000 7f6bf5fb4700 1 --2- 172.21.15.71:0/2877723531 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6bf00f63e0 0x7f6bf0106620 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 rx=0x7f6be00089f0 tx=0x7f6be000b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T13:00:34.164 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:34.161+0000 7f6be67fc700 1 -- 172.21.15.71:0/2877723531 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6be0009320 con 0x7f6bf00f63e0 2024-02-20T13:00:34.164 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:34.161+0000 7f6bf6fb6700 1 -- 172.21.15.71:0/2877723531 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6bf0106d10 con 0x7f6bf00f63e0 2024-02-20T13:00:34.164 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:34.161+0000 7f6be67fc700 1 -- 172.21.15.71:0/2877723531 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(25 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f6be0013ec0 con 0x7f6bf00f63e0 2024-02-20T13:00:34.164 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:34.161+0000 7f6be67fc700 1 -- 172.21.15.71:0/2877723531 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6be0014d30 con 0x7f6bf00f63e0 2024-02-20T13:00:34.164 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:34.161+0000 7f6bf6fb6700 1 -- 172.21.15.71:0/2877723531 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6bf01071a0 con 0x7f6bf00f63e0 2024-02-20T13:00:34.165 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:34.162+0000 7f6be67fc700 1 -- 172.21.15.71:0/2877723531 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 37151+0+0 (secure 0 0 0) 0x7f6be00095f0 con 0x7f6bf00f63e0 2024-02-20T13:00:34.165 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:34.162+0000 7f6bf6fb6700 1 -- 172.21.15.71:0/2877723531 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6bf0045bb0 con 0x7f6bf00f63e0 2024-02-20T13:00:34.165 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:34.162+0000 7f6be67fc700 1 --2- 172.21.15.71:0/2877723531 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f6bdc031090 0x7f6bdc033550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T13:00:34.165 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:34.162+0000 7f6be67fc700 1 -- 172.21.15.71:0/2877723531 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1065+0+0 (secure 0 0 0) 0x7f6be00423e0 con 0x7f6bf00f63e0 2024-02-20T13:00:34.165 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:34.162+0000 7f6bf57b3700 1 --2- 172.21.15.71:0/2877723531 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f6bdc031090 0x7f6bdc033550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T13:00:34.166 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:34.162+0000 7f6bf57b3700 1 --2- 172.21.15.71:0/2877723531 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f6bdc031090 0x7f6bdc033550 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 rx=0x7f6bec0096a0 tx=0x7f6bec006b40).ready entity=mgr.14156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T13:00:34.168 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:34.167+0000 7f6be67fc700 1 -- 172.21.15.71:0/2877723531 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+153237 (secure 0 0 0) 0x7f6be00268c0 con 0x7f6bf00f63e0 2024-02-20T13:00:34.378 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:34.376+0000 7f6bf6fb6700 1 -- 172.21.15.71:0/2877723531 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f6bf0108050 con 0x7f6bf00f63e0 2024-02-20T13:00:34.378 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:34.377+0000 7f6be67fc700 1 -- 172.21.15.71:0/2877723531 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7f6be001d7f0 con 0x7f6bf00f63e0 2024-02-20T13:00:34.379 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-20T13:00:34.379 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":1,"fsid":"680ee126-cfee-11ee-95bb-87774f69a715","modified":"2024-02-20T12:49:44.699059Z","created":"2024-02-20T12:49:44.699059Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi064","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:3300","nonce":0},{"type":"v1","addr":"172.21.15.64:6789","nonce":0}]},"addr":"172.21.15.64:6789/0","public_addr":"172.21.15.64:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-20T13:00:34.380 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:34.380+0000 7f6bf6fb6700 1 -- 172.21.15.71:0/2877723531 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f6bdc031090 msgr2=0x7f6bdc033550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T13:00:34.380 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:34.380+0000 7f6bf6fb6700 1 --2- 172.21.15.71:0/2877723531 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f6bdc031090 0x7f6bdc033550 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 rx=0x7f6bec0096a0 tx=0x7f6bec006b40).stop 2024-02-20T13:00:34.380 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:34.380+0000 7f6bf6fb6700 1 -- 172.21.15.71:0/2877723531 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6bf00f63e0 msgr2=0x7f6bf0106620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T13:00:34.381 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:34.380+0000 7f6bf6fb6700 1 --2- 172.21.15.71:0/2877723531 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6bf00f63e0 0x7f6bf0106620 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 rx=0x7f6be00089f0 tx=0x7f6be000b040).stop 2024-02-20T13:00:34.381 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:34.380+0000 7f6bf6fb6700 1 -- 172.21.15.71:0/2877723531 shutdown_connections 2024-02-20T13:00:34.381 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:34.380+0000 7f6bf6fb6700 1 --2- 172.21.15.71:0/2877723531 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f6bdc031090 0x7f6bdc033550 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T13:00:34.381 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:34.380+0000 7f6bf6fb6700 1 --2- 172.21.15.71:0/2877723531 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6bf00f63e0 0x7f6bf0106620 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T13:00:34.381 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:34.380+0000 7f6bf6fb6700 1 -- 172.21.15.71:0/2877723531 >> 172.21.15.71:0/2877723531 conn(0x7f6bf00f1b40 msgr2=0x7f6bf00f2660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T13:00:34.381 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:34.380+0000 7f6bf6fb6700 1 -- 172.21.15.71:0/2877723531 shutdown_connections 2024-02-20T13:00:34.381 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:34.380+0000 7f6bf6fb6700 1 -- 172.21.15.71:0/2877723531 wait complete. 2024-02-20T13:00:34.381 INFO:teuthology.orchestra.run.smithi071.stderr:dumped monmap epoch 1 2024-02-20T13:00:35.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 13:00:35 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T13:00:33.799010+0000 mgr.smithi064.vriyor (mgr.14156) 301 : cluster [DBG] pgmap v284: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T13:00:35.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 13:00:35 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T13:00:34.379052+0000 mon.smithi064 (mon.0) 279 : audit [DBG] from='client.? 172.21.15.71:0/2877723531' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-20T13:00:36.181 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-20T13:00:36.181 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/daemon-base:latest-pacific shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 680ee126-cfee-11ee-95bb-87774f69a715 -- ceph mon dump -f json 2024-02-20T13:00:37.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 13:00:37 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T13:00:35.799356+0000 mgr.smithi064.vriyor (mgr.14156) 302 : cluster [DBG] pgmap v285: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T13:00:38.322 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:38.315+0000 7f244cd5e700 1 -- 172.21.15.71:0/322125952 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f24480f5480 msgr2=0x7f24480f58a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T13:00:38.322 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:38.315+0000 7f244cd5e700 1 --2- 172.21.15.71:0/322125952 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f24480f5480 0x7f24480f58a0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 rx=0x7f243c009fb0 tx=0x7f243c009b30).stop 2024-02-20T13:00:38.322 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:38.315+0000 7f244cd5e700 1 -- 172.21.15.71:0/322125952 shutdown_connections 2024-02-20T13:00:38.322 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:38.315+0000 7f244cd5e700 1 --2- 172.21.15.71:0/322125952 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f24480f5480 0x7f24480f58a0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T13:00:38.322 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:38.315+0000 7f244cd5e700 1 -- 172.21.15.71:0/322125952 >> 172.21.15.71:0/322125952 conn(0x7f24480f0be0 msgr2=0x7f24480f3000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T13:00:38.322 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:38.315+0000 7f244cd5e700 1 -- 172.21.15.71:0/322125952 shutdown_connections 2024-02-20T13:00:38.322 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:38.315+0000 7f244cd5e700 1 -- 172.21.15.71:0/322125952 wait complete. 2024-02-20T13:00:38.322 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:38.316+0000 7f244cd5e700 1 Processor -- start 2024-02-20T13:00:38.322 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:38.316+0000 7f244cd5e700 1 -- start start 2024-02-20T13:00:38.322 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:38.316+0000 7f244cd5e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f24480f5480 0x7f24481067e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T13:00:38.322 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:38.316+0000 7f244cd5e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2448106cd0 con 0x7f24480f5480 2024-02-20T13:00:38.323 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:38.316+0000 7f244659c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f24480f5480 0x7f24481067e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T13:00:38.323 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:38.316+0000 7f244659c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f24480f5480 0x7f24481067e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.71:59516/0 (socket says 172.21.15.71:59516) 2024-02-20T13:00:38.323 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:38.316+0000 7f244659c700 1 -- 172.21.15.71:0/412936603 learned_addr learned my addr 172.21.15.71:0/412936603 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-02-20T13:00:38.323 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:38.316+0000 7f244659c700 1 -- 172.21.15.71:0/412936603 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f243c005040 con 0x7f24480f5480 2024-02-20T13:00:38.323 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:38.316+0000 7f244659c700 1 --2- 172.21.15.71:0/412936603 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f24480f5480 0x7f24481067e0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 rx=0x7f243c0089f0 tx=0x7f243c00b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T13:00:38.323 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:38.317+0000 7f2436ffd700 1 -- 172.21.15.71:0/412936603 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f243c009320 con 0x7f24480f5480 2024-02-20T13:00:38.323 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:38.317+0000 7f244cd5e700 1 -- 172.21.15.71:0/412936603 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2448106ed0 con 0x7f24480f5480 2024-02-20T13:00:38.323 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:38.317+0000 7f2436ffd700 1 -- 172.21.15.71:0/412936603 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(25 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f243c013ec0 con 0x7f24480f5480 2024-02-20T13:00:38.323 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:38.317+0000 7f2436ffd700 1 -- 172.21.15.71:0/412936603 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f243c014d30 con 0x7f24480f5480 2024-02-20T13:00:38.323 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:38.317+0000 7f244cd5e700 1 -- 172.21.15.71:0/412936603 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2448107360 con 0x7f24480f5480 2024-02-20T13:00:38.323 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:38.318+0000 7f2436ffd700 1 -- 172.21.15.71:0/412936603 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 37151+0+0 (secure 0 0 0) 0x7f243c0095f0 con 0x7f24480f5480 2024-02-20T13:00:38.323 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:38.318+0000 7f244cd5e700 1 -- 172.21.15.71:0/412936603 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2448045bb0 con 0x7f24480f5480 2024-02-20T13:00:38.323 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:38.318+0000 7f2436ffd700 1 --2- 172.21.15.71:0/412936603 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f2430031090 0x7f2430033550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T13:00:38.323 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:38.318+0000 7f2436ffd700 1 -- 172.21.15.71:0/412936603 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1065+0+0 (secure 0 0 0) 0x7f243c0423e0 con 0x7f24480f5480 2024-02-20T13:00:38.324 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:38.318+0000 7f2445d9b700 1 --2- 172.21.15.71:0/412936603 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f2430031090 0x7f2430033550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T13:00:38.324 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:38.319+0000 7f2445d9b700 1 --2- 172.21.15.71:0/412936603 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f2430031090 0x7f2430033550 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 rx=0x7f24380096a0 tx=0x7f2438006b40).ready entity=mgr.14156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T13:00:38.324 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:38.323+0000 7f2436ffd700 1 -- 172.21.15.71:0/412936603 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+153237 (secure 0 0 0) 0x7f243c01d7f0 con 0x7f24480f5480 2024-02-20T13:00:38.531 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:38.530+0000 7f244cd5e700 1 -- 172.21.15.71:0/412936603 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f2448108210 con 0x7f24480f5480 2024-02-20T13:00:38.532 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:38.531+0000 7f2436ffd700 1 -- 172.21.15.71:0/412936603 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7f243c01d7f0 con 0x7f24480f5480 2024-02-20T13:00:38.532 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-20T13:00:38.532 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":1,"fsid":"680ee126-cfee-11ee-95bb-87774f69a715","modified":"2024-02-20T12:49:44.699059Z","created":"2024-02-20T12:49:44.699059Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi064","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:3300","nonce":0},{"type":"v1","addr":"172.21.15.64:6789","nonce":0}]},"addr":"172.21.15.64:6789/0","public_addr":"172.21.15.64:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-20T13:00:38.534 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:38.533+0000 7f244cd5e700 1 -- 172.21.15.71:0/412936603 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f2430031090 msgr2=0x7f2430033550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T13:00:38.534 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:38.533+0000 7f244cd5e700 1 --2- 172.21.15.71:0/412936603 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f2430031090 0x7f2430033550 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 rx=0x7f24380096a0 tx=0x7f2438006b40).stop 2024-02-20T13:00:38.534 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:38.533+0000 7f244cd5e700 1 -- 172.21.15.71:0/412936603 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f24480f5480 msgr2=0x7f24481067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T13:00:38.534 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:38.533+0000 7f244cd5e700 1 --2- 172.21.15.71:0/412936603 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f24480f5480 0x7f24481067e0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 rx=0x7f243c0089f0 tx=0x7f243c00b040).stop 2024-02-20T13:00:38.534 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:38.533+0000 7f244cd5e700 1 -- 172.21.15.71:0/412936603 shutdown_connections 2024-02-20T13:00:38.534 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:38.533+0000 7f244cd5e700 1 --2- 172.21.15.71:0/412936603 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f2430031090 0x7f2430033550 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T13:00:38.535 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:38.533+0000 7f244cd5e700 1 --2- 172.21.15.71:0/412936603 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f24480f5480 0x7f24481067e0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T13:00:38.535 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:38.533+0000 7f244cd5e700 1 -- 172.21.15.71:0/412936603 >> 172.21.15.71:0/412936603 conn(0x7f24480f0be0 msgr2=0x7f244805ff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T13:00:38.535 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:38.533+0000 7f244cd5e700 1 -- 172.21.15.71:0/412936603 shutdown_connections 2024-02-20T13:00:38.535 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:38.533+0000 7f244cd5e700 1 -- 172.21.15.71:0/412936603 wait complete. 2024-02-20T13:00:38.535 INFO:teuthology.orchestra.run.smithi071.stderr:dumped monmap epoch 1 2024-02-20T13:00:39.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 13:00:39 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T13:00:37.799661+0000 mgr.smithi064.vriyor (mgr.14156) 303 : cluster [DBG] pgmap v286: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T13:00:39.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 13:00:39 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T13:00:38.532278+0000 mon.smithi064 (mon.0) 280 : audit [DBG] from='client.? 172.21.15.71:0/412936603' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-20T13:00:40.341 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-20T13:00:40.341 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/daemon-base:latest-pacific shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 680ee126-cfee-11ee-95bb-87774f69a715 -- ceph mon dump -f json 2024-02-20T13:00:41.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 13:00:41 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T13:00:39.799929+0000 mgr.smithi064.vriyor (mgr.14156) 304 : cluster [DBG] pgmap v287: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T13:00:42.280 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:42.278+0000 7fee3be7d700 1 -- 172.21.15.71:0/2808571137 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fee340f54b0 msgr2=0x7fee340f58d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T13:00:42.280 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:42.278+0000 7fee3be7d700 1 --2- 172.21.15.71:0/2808571137 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fee340f54b0 0x7fee340f58d0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 rx=0x7fee24009fb0 tx=0x7fee24009b30).stop 2024-02-20T13:00:42.281 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:42.278+0000 7fee3be7d700 1 -- 172.21.15.71:0/2808571137 shutdown_connections 2024-02-20T13:00:42.281 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:42.278+0000 7fee3be7d700 1 --2- 172.21.15.71:0/2808571137 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fee340f54b0 0x7fee340f58d0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T13:00:42.281 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:42.278+0000 7fee3be7d700 1 -- 172.21.15.71:0/2808571137 >> 172.21.15.71:0/2808571137 conn(0x7fee340f0c10 msgr2=0x7fee340f3030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T13:00:42.281 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:42.278+0000 7fee3be7d700 1 -- 172.21.15.71:0/2808571137 shutdown_connections 2024-02-20T13:00:42.281 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:42.278+0000 7fee3be7d700 1 -- 172.21.15.71:0/2808571137 wait complete. 2024-02-20T13:00:42.281 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:42.278+0000 7fee3be7d700 1 Processor -- start 2024-02-20T13:00:42.281 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:42.279+0000 7fee3be7d700 1 -- start start 2024-02-20T13:00:42.281 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:42.279+0000 7fee3be7d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fee340f54b0 0x7fee341067c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T13:00:42.281 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:42.279+0000 7fee3be7d700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee34106cb0 con 0x7fee340f54b0 2024-02-20T13:00:42.282 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:42.279+0000 7fee39c19700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fee340f54b0 0x7fee341067c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T13:00:42.282 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:42.279+0000 7fee39c19700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fee340f54b0 0x7fee341067c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.71:59524/0 (socket says 172.21.15.71:59524) 2024-02-20T13:00:42.282 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:42.279+0000 7fee39c19700 1 -- 172.21.15.71:0/3022309812 learned_addr learned my addr 172.21.15.71:0/3022309812 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-02-20T13:00:42.282 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:42.279+0000 7fee39c19700 1 -- 172.21.15.71:0/3022309812 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee24005040 con 0x7fee340f54b0 2024-02-20T13:00:42.282 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:42.279+0000 7fee39c19700 1 --2- 172.21.15.71:0/3022309812 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fee340f54b0 0x7fee341067c0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 rx=0x7fee240089f0 tx=0x7fee2400b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T13:00:42.282 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:42.280+0000 7fee2a7fc700 1 -- 172.21.15.71:0/3022309812 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fee24009320 con 0x7fee340f54b0 2024-02-20T13:00:42.283 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:42.280+0000 7fee3be7d700 1 -- 172.21.15.71:0/3022309812 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee34106eb0 con 0x7fee340f54b0 2024-02-20T13:00:42.283 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:42.280+0000 7fee2a7fc700 1 -- 172.21.15.71:0/3022309812 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(25 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7fee24013ec0 con 0x7fee340f54b0 2024-02-20T13:00:42.283 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:42.280+0000 7fee2a7fc700 1 -- 172.21.15.71:0/3022309812 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fee24014d30 con 0x7fee340f54b0 2024-02-20T13:00:42.283 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:42.280+0000 7fee3be7d700 1 -- 172.21.15.71:0/3022309812 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee34107340 con 0x7fee340f54b0 2024-02-20T13:00:42.284 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:42.280+0000 7fee2a7fc700 1 -- 172.21.15.71:0/3022309812 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 37151+0+0 (secure 0 0 0) 0x7fee240095f0 con 0x7fee340f54b0 2024-02-20T13:00:42.284 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:42.281+0000 7fee3be7d700 1 -- 172.21.15.71:0/3022309812 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee34045bb0 con 0x7fee340f54b0 2024-02-20T13:00:42.284 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:42.281+0000 7fee2a7fc700 1 --2- 172.21.15.71:0/3022309812 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fee20031090 0x7fee20033550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T13:00:42.284 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:42.281+0000 7fee2a7fc700 1 -- 172.21.15.71:0/3022309812 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1065+0+0 (secure 0 0 0) 0x7fee240424a0 con 0x7fee340f54b0 2024-02-20T13:00:42.284 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:42.281+0000 7fee39418700 1 --2- 172.21.15.71:0/3022309812 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fee20031090 0x7fee20033550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T13:00:42.284 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:42.281+0000 7fee39418700 1 --2- 172.21.15.71:0/3022309812 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fee20031090 0x7fee20033550 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 rx=0x7fee300096a0 tx=0x7fee30006b40).ready entity=mgr.14156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T13:00:42.286 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:42.286+0000 7fee2a7fc700 1 -- 172.21.15.71:0/3022309812 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+153237 (secure 0 0 0) 0x7fee2400fe40 con 0x7fee340f54b0 2024-02-20T13:00:42.491 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:42.491+0000 7fee3be7d700 1 -- 172.21.15.71:0/3022309812 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fee341081f0 con 0x7fee340f54b0 2024-02-20T13:00:42.492 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:42.491+0000 7fee2a7fc700 1 -- 172.21.15.71:0/3022309812 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7fee2400fe40 con 0x7fee340f54b0 2024-02-20T13:00:42.492 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-20T13:00:42.493 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":1,"fsid":"680ee126-cfee-11ee-95bb-87774f69a715","modified":"2024-02-20T12:49:44.699059Z","created":"2024-02-20T12:49:44.699059Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi064","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:3300","nonce":0},{"type":"v1","addr":"172.21.15.64:6789","nonce":0}]},"addr":"172.21.15.64:6789/0","public_addr":"172.21.15.64:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-20T13:00:42.494 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:42.493+0000 7fee3be7d700 1 -- 172.21.15.71:0/3022309812 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fee20031090 msgr2=0x7fee20033550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T13:00:42.494 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:42.493+0000 7fee3be7d700 1 --2- 172.21.15.71:0/3022309812 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fee20031090 0x7fee20033550 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 rx=0x7fee300096a0 tx=0x7fee30006b40).stop 2024-02-20T13:00:42.495 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:42.493+0000 7fee3be7d700 1 -- 172.21.15.71:0/3022309812 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fee340f54b0 msgr2=0x7fee341067c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T13:00:42.495 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:42.493+0000 7fee3be7d700 1 --2- 172.21.15.71:0/3022309812 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fee340f54b0 0x7fee341067c0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 rx=0x7fee240089f0 tx=0x7fee2400b040).stop 2024-02-20T13:00:42.495 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:42.493+0000 7fee3be7d700 1 -- 172.21.15.71:0/3022309812 shutdown_connections 2024-02-20T13:00:42.495 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:42.493+0000 7fee3be7d700 1 --2- 172.21.15.71:0/3022309812 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fee20031090 0x7fee20033550 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T13:00:42.495 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:42.493+0000 7fee3be7d700 1 --2- 172.21.15.71:0/3022309812 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fee340f54b0 0x7fee341067c0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T13:00:42.495 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:42.493+0000 7fee3be7d700 1 -- 172.21.15.71:0/3022309812 >> 172.21.15.71:0/3022309812 conn(0x7fee340f0c10 msgr2=0x7fee3405ff40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T13:00:42.495 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:42.493+0000 7fee3be7d700 1 -- 172.21.15.71:0/3022309812 shutdown_connections 2024-02-20T13:00:42.495 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:42.493+0000 7fee3be7d700 1 -- 172.21.15.71:0/3022309812 wait complete. 2024-02-20T13:00:42.495 INFO:teuthology.orchestra.run.smithi071.stderr:dumped monmap epoch 1 2024-02-20T13:00:43.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 13:00:43 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T13:00:41.800250+0000 mgr.smithi064.vriyor (mgr.14156) 305 : cluster [DBG] pgmap v288: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T13:00:43.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 13:00:43 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T13:00:42.492255+0000 mon.smithi064 (mon.0) 281 : audit [DBG] from='client.? 172.21.15.71:0/3022309812' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-20T13:00:44.214 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-20T13:00:44.214 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/daemon-base:latest-pacific shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 680ee126-cfee-11ee-95bb-87774f69a715 -- ceph mon dump -f json 2024-02-20T13:00:45.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 13:00:45 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T13:00:43.800590+0000 mgr.smithi064.vriyor (mgr.14156) 306 : cluster [DBG] pgmap v289: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T13:00:46.242 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:46.241+0000 7f5891762700 1 -- 172.21.15.71:0/713309691 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f588c0f5a60 msgr2=0x7f588c0f5e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T13:00:46.242 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:46.241+0000 7f5891762700 1 --2- 172.21.15.71:0/713309691 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f588c0f5a60 0x7f588c0f5e80 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 rx=0x7f587c009fb0 tx=0x7f587c009b30).stop 2024-02-20T13:00:46.243 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:46.242+0000 7f5891762700 1 -- 172.21.15.71:0/713309691 shutdown_connections 2024-02-20T13:00:46.243 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:46.242+0000 7f5891762700 1 --2- 172.21.15.71:0/713309691 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f588c0f5a60 0x7f588c0f5e80 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T13:00:46.243 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:46.242+0000 7f5891762700 1 -- 172.21.15.71:0/713309691 >> 172.21.15.71:0/713309691 conn(0x7f588c0f0390 msgr2=0x7f588c0f27b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T13:00:46.243 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:46.242+0000 7f5891762700 1 -- 172.21.15.71:0/713309691 shutdown_connections 2024-02-20T13:00:46.243 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:46.242+0000 7f5891762700 1 -- 172.21.15.71:0/713309691 wait complete. 2024-02-20T13:00:46.245 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:46.242+0000 7f5891762700 1 Processor -- start 2024-02-20T13:00:46.245 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:46.242+0000 7f5891762700 1 -- start start 2024-02-20T13:00:46.245 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:46.243+0000 7f5891762700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f588c0f5a60 0x7f588c1056d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T13:00:46.245 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:46.243+0000 7f5891762700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f588c105bc0 con 0x7f588c0f5a60 2024-02-20T13:00:46.245 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:46.243+0000 7f588affd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f588c0f5a60 0x7f588c1056d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T13:00:46.245 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:46.243+0000 7f588affd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f588c0f5a60 0x7f588c1056d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.71:59550/0 (socket says 172.21.15.71:59550) 2024-02-20T13:00:46.245 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:46.243+0000 7f588affd700 1 -- 172.21.15.71:0/675211994 learned_addr learned my addr 172.21.15.71:0/675211994 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-02-20T13:00:46.245 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:46.243+0000 7f588affd700 1 -- 172.21.15.71:0/675211994 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f587c005040 con 0x7f588c0f5a60 2024-02-20T13:00:46.245 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:46.243+0000 7f588affd700 1 --2- 172.21.15.71:0/675211994 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f588c0f5a60 0x7f588c1056d0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 rx=0x7f587c0089f0 tx=0x7f587c00b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T13:00:46.246 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:46.243+0000 7f58737fe700 1 -- 172.21.15.71:0/675211994 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f587c009320 con 0x7f588c0f5a60 2024-02-20T13:00:46.246 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:46.243+0000 7f5891762700 1 -- 172.21.15.71:0/675211994 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f588c105dc0 con 0x7f588c0f5a60 2024-02-20T13:00:46.246 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:46.244+0000 7f58737fe700 1 -- 172.21.15.71:0/675211994 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(25 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f587c013ec0 con 0x7f588c0f5a60 2024-02-20T13:00:46.246 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:46.244+0000 7f58737fe700 1 -- 172.21.15.71:0/675211994 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f587c014d30 con 0x7f588c0f5a60 2024-02-20T13:00:46.246 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:46.244+0000 7f5891762700 1 -- 172.21.15.71:0/675211994 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f588c106250 con 0x7f588c0f5a60 2024-02-20T13:00:46.246 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:46.244+0000 7f58737fe700 1 -- 172.21.15.71:0/675211994 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 37151+0+0 (secure 0 0 0) 0x7f587c0095f0 con 0x7f588c0f5a60 2024-02-20T13:00:46.246 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:46.245+0000 7f58737fe700 1 --2- 172.21.15.71:0/675211994 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f5874031090 0x7f5874033550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T13:00:46.246 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:46.244+0000 7f5891762700 1 -- 172.21.15.71:0/675211994 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f588c045bb0 con 0x7f588c0f5a60 2024-02-20T13:00:46.246 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:46.245+0000 7f58737fe700 1 -- 172.21.15.71:0/675211994 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1065+0+0 (secure 0 0 0) 0x7f587c0424a0 con 0x7f588c0f5a60 2024-02-20T13:00:46.246 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:46.245+0000 7f588a7fc700 1 --2- 172.21.15.71:0/675211994 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f5874031090 0x7f5874033550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T13:00:46.247 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:46.245+0000 7f588a7fc700 1 --2- 172.21.15.71:0/675211994 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f5874031090 0x7f5874033550 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 rx=0x7f58800096a0 tx=0x7f5880006b40).ready entity=mgr.14156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T13:00:46.250 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:46.250+0000 7f58737fe700 1 -- 172.21.15.71:0/675211994 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+153237 (secure 0 0 0) 0x7f587c011290 con 0x7f588c0f5a60 2024-02-20T13:00:46.457 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:46.457+0000 7f5891762700 1 -- 172.21.15.71:0/675211994 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f588c107100 con 0x7f588c0f5a60 2024-02-20T13:00:46.458 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:46.457+0000 7f58737fe700 1 -- 172.21.15.71:0/675211994 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7f587c011290 con 0x7f588c0f5a60 2024-02-20T13:00:46.458 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-20T13:00:46.458 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":1,"fsid":"680ee126-cfee-11ee-95bb-87774f69a715","modified":"2024-02-20T12:49:44.699059Z","created":"2024-02-20T12:49:44.699059Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi064","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:3300","nonce":0},{"type":"v1","addr":"172.21.15.64:6789","nonce":0}]},"addr":"172.21.15.64:6789/0","public_addr":"172.21.15.64:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-20T13:00:46.460 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:46.459+0000 7f5891762700 1 -- 172.21.15.71:0/675211994 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f5874031090 msgr2=0x7f5874033550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T13:00:46.460 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:46.459+0000 7f5891762700 1 --2- 172.21.15.71:0/675211994 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f5874031090 0x7f5874033550 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 rx=0x7f58800096a0 tx=0x7f5880006b40).stop 2024-02-20T13:00:46.460 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:46.459+0000 7f5891762700 1 -- 172.21.15.71:0/675211994 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f588c0f5a60 msgr2=0x7f588c1056d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T13:00:46.460 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:46.459+0000 7f5891762700 1 --2- 172.21.15.71:0/675211994 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f588c0f5a60 0x7f588c1056d0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 rx=0x7f587c0089f0 tx=0x7f587c00b040).stop 2024-02-20T13:00:46.461 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:46.459+0000 7f5891762700 1 -- 172.21.15.71:0/675211994 shutdown_connections 2024-02-20T13:00:46.461 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:46.459+0000 7f5891762700 1 --2- 172.21.15.71:0/675211994 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f5874031090 0x7f5874033550 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T13:00:46.461 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:46.459+0000 7f5891762700 1 --2- 172.21.15.71:0/675211994 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f588c0f5a60 0x7f588c1056d0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T13:00:46.461 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:46.459+0000 7f5891762700 1 -- 172.21.15.71:0/675211994 >> 172.21.15.71:0/675211994 conn(0x7f588c0f0390 msgr2=0x7f588c0f0e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T13:00:46.461 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:46.459+0000 7f5891762700 1 -- 172.21.15.71:0/675211994 shutdown_connections 2024-02-20T13:00:46.461 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:46.459+0000 7f5891762700 1 -- 172.21.15.71:0/675211994 wait complete. 2024-02-20T13:00:46.461 INFO:teuthology.orchestra.run.smithi071.stderr:dumped monmap epoch 1 2024-02-20T13:00:47.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 13:00:47 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T13:00:45.800954+0000 mgr.smithi064.vriyor (mgr.14156) 307 : cluster [DBG] pgmap v290: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T13:00:47.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 13:00:47 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T13:00:46.458421+0000 mon.smithi064 (mon.0) 282 : audit [DBG] from='client.? 172.21.15.71:0/675211994' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-20T13:00:48.325 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-20T13:00:48.325 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/daemon-base:latest-pacific shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 680ee126-cfee-11ee-95bb-87774f69a715 -- ceph mon dump -f json 2024-02-20T13:00:49.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 13:00:49 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T13:00:47.801311+0000 mgr.smithi064.vriyor (mgr.14156) 308 : cluster [DBG] pgmap v291: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T13:00:50.242 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:50.240+0000 7effac8a1700 1 -- 172.21.15.71:0/752930289 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7effa80f6590 msgr2=0x7effa80f69b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T13:00:50.242 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:50.240+0000 7effac8a1700 1 --2- 172.21.15.71:0/752930289 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7effa80f6590 0x7effa80f69b0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 rx=0x7eff9c004760 tx=0x7eff9c009ad0).stop 2024-02-20T13:00:50.242 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:50.240+0000 7effac8a1700 1 -- 172.21.15.71:0/752930289 shutdown_connections 2024-02-20T13:00:50.242 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:50.240+0000 7effac8a1700 1 --2- 172.21.15.71:0/752930289 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7effa80f6590 0x7effa80f69b0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T13:00:50.243 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:50.240+0000 7effac8a1700 1 -- 172.21.15.71:0/752930289 >> 172.21.15.71:0/752930289 conn(0x7effa80f1cd0 msgr2=0x7effa80f4110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T13:00:50.243 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:50.240+0000 7effac8a1700 1 -- 172.21.15.71:0/752930289 shutdown_connections 2024-02-20T13:00:50.243 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:50.240+0000 7effac8a1700 1 -- 172.21.15.71:0/752930289 wait complete. 2024-02-20T13:00:50.243 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:50.241+0000 7effac8a1700 1 Processor -- start 2024-02-20T13:00:50.243 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:50.241+0000 7effac8a1700 1 -- start start 2024-02-20T13:00:50.243 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:50.241+0000 7effac8a1700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7effa80f6590 0x7effa806ba80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T13:00:50.243 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:50.241+0000 7effac8a1700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effa8069f00 con 0x7effa80f6590 2024-02-20T13:00:50.243 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:50.241+0000 7effa659c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7effa80f6590 0x7effa806ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T13:00:50.243 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:50.241+0000 7effa659c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7effa80f6590 0x7effa806ba80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.71:56834/0 (socket says 172.21.15.71:56834) 2024-02-20T13:00:50.243 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:50.241+0000 7effa659c700 1 -- 172.21.15.71:0/2981089330 learned_addr learned my addr 172.21.15.71:0/2981089330 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-02-20T13:00:50.243 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:50.242+0000 7effa659c700 1 -- 172.21.15.71:0/2981089330 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff9c009d70 con 0x7effa80f6590 2024-02-20T13:00:50.243 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:50.242+0000 7effa659c700 1 --2- 172.21.15.71:0/2981089330 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7effa80f6590 0x7effa806ba80 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 rx=0x7eff9c004170 tx=0x7eff9c009ad0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T13:00:50.244 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:50.242+0000 7eff96ffd700 1 -- 172.21.15.71:0/2981089330 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7eff9c00e040 con 0x7effa80f6590 2024-02-20T13:00:50.244 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:50.242+0000 7eff96ffd700 1 -- 172.21.15.71:0/2981089330 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(25 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7eff9c008ef0 con 0x7effa80f6590 2024-02-20T13:00:50.245 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:50.242+0000 7effac8a1700 1 -- 172.21.15.71:0/2981089330 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7effa806a0a0 con 0x7effa80f6590 2024-02-20T13:00:50.245 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:50.242+0000 7eff96ffd700 1 -- 172.21.15.71:0/2981089330 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7eff9c014c90 con 0x7effa80f6590 2024-02-20T13:00:50.245 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:50.242+0000 7effac8a1700 1 -- 172.21.15.71:0/2981089330 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7effa806a530 con 0x7effa80f6590 2024-02-20T13:00:50.245 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:50.243+0000 7eff96ffd700 1 -- 172.21.15.71:0/2981089330 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 37151+0+0 (secure 0 0 0) 0x7eff9c014420 con 0x7effa80f6590 2024-02-20T13:00:50.245 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:50.243+0000 7eff96ffd700 1 --2- 172.21.15.71:0/2981089330 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7eff90031090 0x7eff90033550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T13:00:50.245 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:50.243+0000 7eff96ffd700 1 -- 172.21.15.71:0/2981089330 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1065+0+0 (secure 0 0 0) 0x7eff9c0421f0 con 0x7effa80f6590 2024-02-20T13:00:50.245 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:50.243+0000 7effac8a1700 1 -- 172.21.15.71:0/2981089330 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7effa80591e0 con 0x7effa80f6590 2024-02-20T13:00:50.245 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:50.243+0000 7effa5d9b700 1 --2- 172.21.15.71:0/2981089330 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7eff90031090 0x7eff90033550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T13:00:50.246 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:50.243+0000 7effa5d9b700 1 --2- 172.21.15.71:0/2981089330 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7eff90031090 0x7eff90033550 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 rx=0x7eff980096a0 tx=0x7eff98006b40).ready entity=mgr.14156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T13:00:50.248 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:50.248+0000 7eff96ffd700 1 -- 172.21.15.71:0/2981089330 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+153237 (secure 0 0 0) 0x7eff9c026300 con 0x7effa80f6590 2024-02-20T13:00:50.456 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:50.456+0000 7effac8a1700 1 -- 172.21.15.71:0/2981089330 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7effa8045bb0 con 0x7effa80f6590 2024-02-20T13:00:50.457 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:50.456+0000 7eff96ffd700 1 -- 172.21.15.71:0/2981089330 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7eff9c012b20 con 0x7effa80f6590 2024-02-20T13:00:50.457 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-20T13:00:50.457 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":1,"fsid":"680ee126-cfee-11ee-95bb-87774f69a715","modified":"2024-02-20T12:49:44.699059Z","created":"2024-02-20T12:49:44.699059Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi064","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:3300","nonce":0},{"type":"v1","addr":"172.21.15.64:6789","nonce":0}]},"addr":"172.21.15.64:6789/0","public_addr":"172.21.15.64:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-20T13:00:50.458 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:50.458+0000 7effac8a1700 1 -- 172.21.15.71:0/2981089330 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7eff90031090 msgr2=0x7eff90033550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T13:00:50.458 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:50.458+0000 7effac8a1700 1 --2- 172.21.15.71:0/2981089330 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7eff90031090 0x7eff90033550 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 rx=0x7eff980096a0 tx=0x7eff98006b40).stop 2024-02-20T13:00:50.459 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:50.458+0000 7effac8a1700 1 -- 172.21.15.71:0/2981089330 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7effa80f6590 msgr2=0x7effa806ba80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T13:00:50.459 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:50.458+0000 7effac8a1700 1 --2- 172.21.15.71:0/2981089330 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7effa80f6590 0x7effa806ba80 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 rx=0x7eff9c004170 tx=0x7eff9c009ad0).stop 2024-02-20T13:00:50.459 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:50.458+0000 7effac8a1700 1 -- 172.21.15.71:0/2981089330 shutdown_connections 2024-02-20T13:00:50.459 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:50.458+0000 7effac8a1700 1 --2- 172.21.15.71:0/2981089330 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7eff90031090 0x7eff90033550 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T13:00:50.459 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:50.458+0000 7effac8a1700 1 --2- 172.21.15.71:0/2981089330 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7effa80f6590 0x7effa806ba80 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T13:00:50.459 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:50.458+0000 7effac8a1700 1 -- 172.21.15.71:0/2981089330 >> 172.21.15.71:0/2981089330 conn(0x7effa80f1cd0 msgr2=0x7effa80f27c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T13:00:50.459 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:50.458+0000 7effac8a1700 1 -- 172.21.15.71:0/2981089330 shutdown_connections 2024-02-20T13:00:50.459 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:50.458+0000 7effac8a1700 1 -- 172.21.15.71:0/2981089330 wait complete. 2024-02-20T13:00:50.459 INFO:teuthology.orchestra.run.smithi071.stderr:dumped monmap epoch 1 2024-02-20T13:00:51.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 13:00:51 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T13:00:49.801653+0000 mgr.smithi064.vriyor (mgr.14156) 309 : cluster [DBG] pgmap v292: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T13:00:51.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 13:00:51 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T13:00:50.457241+0000 mon.smithi064 (mon.0) 283 : audit [DBG] from='client.? 172.21.15.71:0/2981089330' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-20T13:00:52.209 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-20T13:00:52.209 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/daemon-base:latest-pacific shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 680ee126-cfee-11ee-95bb-87774f69a715 -- ceph mon dump -f json 2024-02-20T13:00:53.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 13:00:53 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T13:00:51.802041+0000 mgr.smithi064.vriyor (mgr.14156) 310 : cluster [DBG] pgmap v293: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T13:00:54.140 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:54.133+0000 7f26f95d9700 1 -- 172.21.15.71:0/3572032661 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f26f40f6590 msgr2=0x7f26f40f69b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T13:00:54.141 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:54.133+0000 7f26f95d9700 1 --2- 172.21.15.71:0/3572032661 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f26f40f6590 0x7f26f40f69b0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 rx=0x7f26e4009fb0 tx=0x7f26e4009b30).stop 2024-02-20T13:00:54.141 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:54.133+0000 7f26f95d9700 1 -- 172.21.15.71:0/3572032661 shutdown_connections 2024-02-20T13:00:54.141 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:54.133+0000 7f26f95d9700 1 --2- 172.21.15.71:0/3572032661 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f26f40f6590 0x7f26f40f69b0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T13:00:54.141 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:54.133+0000 7f26f95d9700 1 -- 172.21.15.71:0/3572032661 >> 172.21.15.71:0/3572032661 conn(0x7f26f40f1cf0 msgr2=0x7f26f40f4110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T13:00:54.141 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:54.133+0000 7f26f95d9700 1 -- 172.21.15.71:0/3572032661 shutdown_connections 2024-02-20T13:00:54.141 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:54.133+0000 7f26f95d9700 1 -- 172.21.15.71:0/3572032661 wait complete. 2024-02-20T13:00:54.141 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:54.133+0000 7f26f95d9700 1 Processor -- start 2024-02-20T13:00:54.141 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:54.133+0000 7f26f95d9700 1 -- start start 2024-02-20T13:00:54.141 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:54.133+0000 7f26f95d9700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f26f40f6590 0x7f26f4106780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T13:00:54.141 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:54.133+0000 7f26f95d9700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26f4106c70 con 0x7f26f40f6590 2024-02-20T13:00:54.141 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:54.134+0000 7f26f2ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f26f40f6590 0x7f26f4106780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T13:00:54.141 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:54.134+0000 7f26f2ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f26f40f6590 0x7f26f4106780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.71:56844/0 (socket says 172.21.15.71:56844) 2024-02-20T13:00:54.141 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:54.134+0000 7f26f2ffd700 1 -- 172.21.15.71:0/847028449 learned_addr learned my addr 172.21.15.71:0/847028449 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-02-20T13:00:54.142 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:54.134+0000 7f26f2ffd700 1 -- 172.21.15.71:0/847028449 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26e4005040 con 0x7f26f40f6590 2024-02-20T13:00:54.142 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:54.134+0000 7f26f2ffd700 1 --2- 172.21.15.71:0/847028449 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f26f40f6590 0x7f26f4106780 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 rx=0x7f26e40089f0 tx=0x7f26e400b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T13:00:54.142 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:54.134+0000 7f26db7fe700 1 -- 172.21.15.71:0/847028449 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f26e4009320 con 0x7f26f40f6590 2024-02-20T13:00:54.142 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:54.134+0000 7f26f95d9700 1 -- 172.21.15.71:0/847028449 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26f4106e70 con 0x7f26f40f6590 2024-02-20T13:00:54.142 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:54.134+0000 7f26db7fe700 1 -- 172.21.15.71:0/847028449 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(25 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f26e4013ec0 con 0x7f26f40f6590 2024-02-20T13:00:54.142 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:54.134+0000 7f26db7fe700 1 -- 172.21.15.71:0/847028449 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f26e4014d30 con 0x7f26f40f6590 2024-02-20T13:00:54.142 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:54.134+0000 7f26f95d9700 1 -- 172.21.15.71:0/847028449 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26f4107300 con 0x7f26f40f6590 2024-02-20T13:00:54.142 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:54.135+0000 7f26db7fe700 1 -- 172.21.15.71:0/847028449 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 37151+0+0 (secure 0 0 0) 0x7f26e40095f0 con 0x7f26f40f6590 2024-02-20T13:00:54.142 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:54.135+0000 7f26f95d9700 1 -- 172.21.15.71:0/847028449 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26f4045bb0 con 0x7f26f40f6590 2024-02-20T13:00:54.142 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:54.135+0000 7f26db7fe700 1 --2- 172.21.15.71:0/847028449 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f26dc031090 0x7f26dc033550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T13:00:54.142 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:54.135+0000 7f26db7fe700 1 -- 172.21.15.71:0/847028449 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1065+0+0 (secure 0 0 0) 0x7f26e40423e0 con 0x7f26f40f6590 2024-02-20T13:00:54.142 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:54.135+0000 7f26f27fc700 1 --2- 172.21.15.71:0/847028449 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f26dc031090 0x7f26dc033550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T13:00:54.142 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:54.136+0000 7f26f27fc700 1 --2- 172.21.15.71:0/847028449 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f26dc031090 0x7f26dc033550 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 rx=0x7f26e80096a0 tx=0x7f26e8006b40).ready entity=mgr.14156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T13:00:54.142 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:54.140+0000 7f26db7fe700 1 -- 172.21.15.71:0/847028449 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+153237 (secure 0 0 0) 0x7f26e40263a0 con 0x7f26f40f6590 2024-02-20T13:00:54.348 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:54.347+0000 7f26f95d9700 1 -- 172.21.15.71:0/847028449 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f26f41081b0 con 0x7f26f40f6590 2024-02-20T13:00:54.348 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:54.348+0000 7f26db7fe700 1 -- 172.21.15.71:0/847028449 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7f26e401d7f0 con 0x7f26f40f6590 2024-02-20T13:00:54.348 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-20T13:00:54.348 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":1,"fsid":"680ee126-cfee-11ee-95bb-87774f69a715","modified":"2024-02-20T12:49:44.699059Z","created":"2024-02-20T12:49:44.699059Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi064","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:3300","nonce":0},{"type":"v1","addr":"172.21.15.64:6789","nonce":0}]},"addr":"172.21.15.64:6789/0","public_addr":"172.21.15.64:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-20T13:00:54.350 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:54.349+0000 7f26f95d9700 1 -- 172.21.15.71:0/847028449 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f26dc031090 msgr2=0x7f26dc033550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T13:00:54.350 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:54.349+0000 7f26f95d9700 1 --2- 172.21.15.71:0/847028449 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f26dc031090 0x7f26dc033550 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 rx=0x7f26e80096a0 tx=0x7f26e8006b40).stop 2024-02-20T13:00:54.350 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:54.349+0000 7f26f95d9700 1 -- 172.21.15.71:0/847028449 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f26f40f6590 msgr2=0x7f26f4106780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T13:00:54.350 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:54.349+0000 7f26f95d9700 1 --2- 172.21.15.71:0/847028449 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f26f40f6590 0x7f26f4106780 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 rx=0x7f26e40089f0 tx=0x7f26e400b040).stop 2024-02-20T13:00:54.350 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:54.349+0000 7f26f95d9700 1 -- 172.21.15.71:0/847028449 shutdown_connections 2024-02-20T13:00:54.350 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:54.349+0000 7f26f95d9700 1 --2- 172.21.15.71:0/847028449 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f26dc031090 0x7f26dc033550 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T13:00:54.350 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:54.349+0000 7f26f95d9700 1 --2- 172.21.15.71:0/847028449 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f26f40f6590 0x7f26f4106780 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T13:00:54.350 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:54.349+0000 7f26f95d9700 1 -- 172.21.15.71:0/847028449 >> 172.21.15.71:0/847028449 conn(0x7f26f40f1cf0 msgr2=0x7f26f40f27e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T13:00:54.351 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:54.349+0000 7f26f95d9700 1 -- 172.21.15.71:0/847028449 shutdown_connections 2024-02-20T13:00:54.351 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:54.349+0000 7f26f95d9700 1 -- 172.21.15.71:0/847028449 wait complete. 2024-02-20T13:00:54.351 INFO:teuthology.orchestra.run.smithi071.stderr:dumped monmap epoch 1 2024-02-20T13:00:54.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 13:00:54 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T13:00:53.853232+0000 mon.smithi064 (mon.0) 284 : audit [INF] from='mgr.14156 172.21.15.64:0/3284974785' entity='mgr.smithi064.vriyor' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi064.vriyor/mirror_snapshot_schedule"}]: dispatch 2024-02-20T13:00:54.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 13:00:54 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T13:00:53.853456+0000 mon.smithi064 (mon.0) 285 : audit [INF] from='mgr.14156 172.21.15.64:0/3284974785' entity='mgr.smithi064.vriyor' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi064.vriyor/trash_purge_schedule"}]: dispatch 2024-02-20T13:00:55.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 13:00:55 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T13:00:53.802394+0000 mgr.smithi064.vriyor (mgr.14156) 311 : cluster [DBG] pgmap v294: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T13:00:55.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 13:00:55 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T13:00:54.348725+0000 mon.smithi064 (mon.0) 286 : audit [DBG] from='client.? 172.21.15.71:0/847028449' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-20T13:00:56.371 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-20T13:00:56.372 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/daemon-base:latest-pacific shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 680ee126-cfee-11ee-95bb-87774f69a715 -- ceph mon dump -f json 2024-02-20T13:00:57.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 13:00:57 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T13:00:55.802699+0000 mgr.smithi064.vriyor (mgr.14156) 312 : cluster [DBG] pgmap v295: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T13:00:58.435 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:58.434+0000 7fe1955a6700 1 -- 172.21.15.71:0/793567507 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe1900f54b0 msgr2=0x7fe1900f58d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T13:00:58.435 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:58.434+0000 7fe1955a6700 1 --2- 172.21.15.71:0/793567507 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe1900f54b0 0x7fe1900f58d0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 rx=0x7fe180009fb0 tx=0x7fe180009b30).stop 2024-02-20T13:00:58.435 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:58.434+0000 7fe1955a6700 1 -- 172.21.15.71:0/793567507 shutdown_connections 2024-02-20T13:00:58.435 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:58.434+0000 7fe1955a6700 1 --2- 172.21.15.71:0/793567507 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe1900f54b0 0x7fe1900f58d0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T13:00:58.435 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:58.434+0000 7fe1955a6700 1 -- 172.21.15.71:0/793567507 >> 172.21.15.71:0/793567507 conn(0x7fe1900f0c10 msgr2=0x7fe1900f3030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T13:00:58.435 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:58.434+0000 7fe1955a6700 1 -- 172.21.15.71:0/793567507 shutdown_connections 2024-02-20T13:00:58.436 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:58.435+0000 7fe1955a6700 1 -- 172.21.15.71:0/793567507 wait complete. 2024-02-20T13:00:58.437 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:58.435+0000 7fe1955a6700 1 Processor -- start 2024-02-20T13:00:58.437 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:58.435+0000 7fe1955a6700 1 -- start start 2024-02-20T13:00:58.437 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:58.435+0000 7fe1955a6700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe1900f54b0 0x7fe1901094c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T13:00:58.437 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:58.435+0000 7fe1955a6700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe190060300 con 0x7fe1900f54b0 2024-02-20T13:00:58.437 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:58.436+0000 7fe18effd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe1900f54b0 0x7fe1901094c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T13:00:58.437 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:58.436+0000 7fe18effd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe1900f54b0 0x7fe1901094c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.71:56506/0 (socket says 172.21.15.71:56506) 2024-02-20T13:00:58.438 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:58.436+0000 7fe18effd700 1 -- 172.21.15.71:0/594020509 learned_addr learned my addr 172.21.15.71:0/594020509 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-02-20T13:00:58.438 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:58.436+0000 7fe18effd700 1 -- 172.21.15.71:0/594020509 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe180005040 con 0x7fe1900f54b0 2024-02-20T13:00:58.438 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:58.436+0000 7fe18effd700 1 --2- 172.21.15.71:0/594020509 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe1900f54b0 0x7fe1901094c0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 rx=0x7fe1800089f0 tx=0x7fe18000b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T13:00:58.438 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:58.436+0000 7fe1877fe700 1 -- 172.21.15.71:0/594020509 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe180009320 con 0x7fe1900f54b0 2024-02-20T13:00:58.438 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:58.436+0000 7fe1877fe700 1 -- 172.21.15.71:0/594020509 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(25 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7fe180013ec0 con 0x7fe1900f54b0 2024-02-20T13:00:58.438 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:58.436+0000 7fe1955a6700 1 -- 172.21.15.71:0/594020509 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe190060500 con 0x7fe1900f54b0 2024-02-20T13:00:58.438 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:58.436+0000 7fe1877fe700 1 -- 172.21.15.71:0/594020509 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe180014d30 con 0x7fe1900f54b0 2024-02-20T13:00:58.438 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:58.437+0000 7fe1955a6700 1 -- 172.21.15.71:0/594020509 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe190109b70 con 0x7fe1900f54b0 2024-02-20T13:00:58.439 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:58.437+0000 7fe1877fe700 1 -- 172.21.15.71:0/594020509 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 37151+0+0 (secure 0 0 0) 0x7fe1800095f0 con 0x7fe1900f54b0 2024-02-20T13:00:58.439 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:58.437+0000 7fe1955a6700 1 -- 172.21.15.71:0/594020509 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1900591e0 con 0x7fe1900f54b0 2024-02-20T13:00:58.439 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:58.437+0000 7fe1877fe700 1 --2- 172.21.15.71:0/594020509 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fe17c0310e0 0x7fe17c0335a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T13:00:58.439 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:58.438+0000 7fe1877fe700 1 -- 172.21.15.71:0/594020509 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1065+0+0 (secure 0 0 0) 0x7fe180042420 con 0x7fe1900f54b0 2024-02-20T13:00:58.439 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:58.438+0000 7fe18e7fc700 1 --2- 172.21.15.71:0/594020509 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fe17c0310e0 0x7fe17c0335a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T13:00:58.439 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:58.438+0000 7fe18e7fc700 1 --2- 172.21.15.71:0/594020509 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fe17c0310e0 0x7fe17c0335a0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 rx=0x7fe1780096a0 tx=0x7fe178006b40).ready entity=mgr.14156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T13:00:58.442 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:58.442+0000 7fe1877fe700 1 -- 172.21.15.71:0/594020509 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+153237 (secure 0 0 0) 0x7fe18001d7f0 con 0x7fe1900f54b0 2024-02-20T13:00:58.648 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:58.647+0000 7fe1955a6700 1 -- 172.21.15.71:0/594020509 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fe1900fafa0 con 0x7fe1900f54b0 2024-02-20T13:00:58.648 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:58.648+0000 7fe1877fe700 1 -- 172.21.15.71:0/594020509 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7fe18000fe40 con 0x7fe1900f54b0 2024-02-20T13:00:58.649 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-20T13:00:58.649 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":1,"fsid":"680ee126-cfee-11ee-95bb-87774f69a715","modified":"2024-02-20T12:49:44.699059Z","created":"2024-02-20T12:49:44.699059Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi064","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:3300","nonce":0},{"type":"v1","addr":"172.21.15.64:6789","nonce":0}]},"addr":"172.21.15.64:6789/0","public_addr":"172.21.15.64:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-20T13:00:58.650 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:58.650+0000 7fe1955a6700 1 -- 172.21.15.71:0/594020509 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fe17c0310e0 msgr2=0x7fe17c0335a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T13:00:58.650 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:58.650+0000 7fe1955a6700 1 --2- 172.21.15.71:0/594020509 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fe17c0310e0 0x7fe17c0335a0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 rx=0x7fe1780096a0 tx=0x7fe178006b40).stop 2024-02-20T13:00:58.650 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:58.650+0000 7fe1955a6700 1 -- 172.21.15.71:0/594020509 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe1900f54b0 msgr2=0x7fe1901094c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T13:00:58.650 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:58.650+0000 7fe1955a6700 1 --2- 172.21.15.71:0/594020509 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe1900f54b0 0x7fe1901094c0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 rx=0x7fe1800089f0 tx=0x7fe18000b040).stop 2024-02-20T13:00:58.650 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:58.650+0000 7fe1955a6700 1 -- 172.21.15.71:0/594020509 shutdown_connections 2024-02-20T13:00:58.650 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:58.650+0000 7fe1955a6700 1 --2- 172.21.15.71:0/594020509 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fe17c0310e0 0x7fe17c0335a0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T13:00:58.650 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:58.650+0000 7fe1955a6700 1 --2- 172.21.15.71:0/594020509 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe1900f54b0 0x7fe1901094c0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T13:00:58.651 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:58.650+0000 7fe1955a6700 1 -- 172.21.15.71:0/594020509 >> 172.21.15.71:0/594020509 conn(0x7fe1900f0c10 msgr2=0x7fe1900f1670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T13:00:58.651 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:58.650+0000 7fe1955a6700 1 -- 172.21.15.71:0/594020509 shutdown_connections 2024-02-20T13:00:58.651 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:00:58.650+0000 7fe1955a6700 1 -- 172.21.15.71:0/594020509 wait complete. 2024-02-20T13:00:58.651 INFO:teuthology.orchestra.run.smithi071.stderr:dumped monmap epoch 1 2024-02-20T13:00:59.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 13:00:59 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T13:00:57.803063+0000 mgr.smithi064.vriyor (mgr.14156) 313 : cluster [DBG] pgmap v296: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T13:00:59.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 13:00:59 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T13:00:58.648941+0000 mon.smithi064 (mon.0) 287 : audit [DBG] from='client.? 172.21.15.71:0/594020509' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-20T13:01:00.288 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-20T13:01:00.289 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/daemon-base:latest-pacific shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 680ee126-cfee-11ee-95bb-87774f69a715 -- ceph mon dump -f json 2024-02-20T13:01:01.405 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 13:01:01 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T13:00:59.803448+0000 mgr.smithi064.vriyor (mgr.14156) 314 : cluster [DBG] pgmap v297: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T13:01:02.252 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:02.250+0000 7f0a65771700 1 -- 172.21.15.71:0/1397125120 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0a600f6590 msgr2=0x7f0a600f69b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T13:01:02.252 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:02.250+0000 7f0a65771700 1 --2- 172.21.15.71:0/1397125120 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0a600f6590 0x7f0a600f69b0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 rx=0x7f0a54004760 tx=0x7f0a54009ad0).stop 2024-02-20T13:01:02.252 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:02.250+0000 7f0a65771700 1 -- 172.21.15.71:0/1397125120 shutdown_connections 2024-02-20T13:01:02.253 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:02.250+0000 7f0a65771700 1 --2- 172.21.15.71:0/1397125120 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0a600f6590 0x7f0a600f69b0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T13:01:02.253 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:02.250+0000 7f0a65771700 1 -- 172.21.15.71:0/1397125120 >> 172.21.15.71:0/1397125120 conn(0x7f0a600f1cd0 msgr2=0x7f0a600f4110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T13:01:02.253 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:02.250+0000 7f0a65771700 1 -- 172.21.15.71:0/1397125120 shutdown_connections 2024-02-20T13:01:02.253 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:02.250+0000 7f0a65771700 1 -- 172.21.15.71:0/1397125120 wait complete. 2024-02-20T13:01:02.253 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:02.251+0000 7f0a65771700 1 Processor -- start 2024-02-20T13:01:02.253 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:02.251+0000 7f0a65771700 1 -- start start 2024-02-20T13:01:02.253 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:02.251+0000 7f0a65771700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0a600f6590 0x7f0a601067c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T13:01:02.253 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:02.251+0000 7f0a65771700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a60106cb0 con 0x7f0a600f6590 2024-02-20T13:01:02.253 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:02.251+0000 7f0a5effd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0a600f6590 0x7f0a601067c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T13:01:02.253 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:02.251+0000 7f0a5effd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0a600f6590 0x7f0a601067c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.71:56524/0 (socket says 172.21.15.71:56524) 2024-02-20T13:01:02.253 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:02.251+0000 7f0a5effd700 1 -- 172.21.15.71:0/285005275 learned_addr learned my addr 172.21.15.71:0/285005275 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-02-20T13:01:02.253 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:02.251+0000 7f0a5effd700 1 -- 172.21.15.71:0/285005275 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a54009d70 con 0x7f0a600f6590 2024-02-20T13:01:02.253 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:02.251+0000 7f0a5effd700 1 --2- 172.21.15.71:0/285005275 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0a600f6590 0x7f0a601067c0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 rx=0x7f0a54004170 tx=0x7f0a54009ad0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T13:01:02.254 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:02.252+0000 7f0a477fe700 1 -- 172.21.15.71:0/285005275 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0a5400e040 con 0x7f0a600f6590 2024-02-20T13:01:02.254 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:02.252+0000 7f0a65771700 1 -- 172.21.15.71:0/285005275 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a60106eb0 con 0x7f0a600f6590 2024-02-20T13:01:02.254 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:02.252+0000 7f0a477fe700 1 -- 172.21.15.71:0/285005275 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(25 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f0a54008ef0 con 0x7f0a600f6590 2024-02-20T13:01:02.254 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:02.252+0000 7f0a477fe700 1 -- 172.21.15.71:0/285005275 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0a54014c90 con 0x7f0a600f6590 2024-02-20T13:01:02.254 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:02.252+0000 7f0a65771700 1 -- 172.21.15.71:0/285005275 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a60107340 con 0x7f0a600f6590 2024-02-20T13:01:02.255 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:02.252+0000 7f0a477fe700 1 -- 172.21.15.71:0/285005275 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 37151+0+0 (secure 0 0 0) 0x7f0a54014420 con 0x7f0a600f6590 2024-02-20T13:01:02.255 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:02.253+0000 7f0a65771700 1 -- 172.21.15.71:0/285005275 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a60045bb0 con 0x7f0a600f6590 2024-02-20T13:01:02.255 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:02.253+0000 7f0a477fe700 1 --2- 172.21.15.71:0/285005275 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f0a48031090 0x7f0a48033550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T13:01:02.255 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:02.253+0000 7f0a477fe700 1 -- 172.21.15.71:0/285005275 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1065+0+0 (secure 0 0 0) 0x7f0a540421f0 con 0x7f0a600f6590 2024-02-20T13:01:02.255 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:02.253+0000 7f0a5e7fc700 1 --2- 172.21.15.71:0/285005275 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f0a48031090 0x7f0a48033550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T13:01:02.255 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:02.253+0000 7f0a5e7fc700 1 --2- 172.21.15.71:0/285005275 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f0a48031090 0x7f0a48033550 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 rx=0x7f0a500096a0 tx=0x7f0a50006b40).ready entity=mgr.14156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T13:01:02.258 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:02.258+0000 7f0a477fe700 1 -- 172.21.15.71:0/285005275 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+153237 (secure 0 0 0) 0x7f0a54028c90 con 0x7f0a600f6590 2024-02-20T13:01:02.464 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:02.463+0000 7f0a65771700 1 -- 172.21.15.71:0/285005275 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f0a601081f0 con 0x7f0a600f6590 2024-02-20T13:01:02.464 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:02.464+0000 7f0a477fe700 1 -- 172.21.15.71:0/285005275 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7f0a54012b20 con 0x7f0a600f6590 2024-02-20T13:01:02.464 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-20T13:01:02.465 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":1,"fsid":"680ee126-cfee-11ee-95bb-87774f69a715","modified":"2024-02-20T12:49:44.699059Z","created":"2024-02-20T12:49:44.699059Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi064","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:3300","nonce":0},{"type":"v1","addr":"172.21.15.64:6789","nonce":0}]},"addr":"172.21.15.64:6789/0","public_addr":"172.21.15.64:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-20T13:01:02.466 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:02.465+0000 7f0a65771700 1 -- 172.21.15.71:0/285005275 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f0a48031090 msgr2=0x7f0a48033550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T13:01:02.467 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:02.465+0000 7f0a65771700 1 --2- 172.21.15.71:0/285005275 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f0a48031090 0x7f0a48033550 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 rx=0x7f0a500096a0 tx=0x7f0a50006b40).stop 2024-02-20T13:01:02.467 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:02.465+0000 7f0a65771700 1 -- 172.21.15.71:0/285005275 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0a600f6590 msgr2=0x7f0a601067c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T13:01:02.467 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:02.465+0000 7f0a65771700 1 --2- 172.21.15.71:0/285005275 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0a600f6590 0x7f0a601067c0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 rx=0x7f0a54004170 tx=0x7f0a54009ad0).stop 2024-02-20T13:01:02.467 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:02.465+0000 7f0a65771700 1 -- 172.21.15.71:0/285005275 shutdown_connections 2024-02-20T13:01:02.467 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:02.465+0000 7f0a65771700 1 --2- 172.21.15.71:0/285005275 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f0a48031090 0x7f0a48033550 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T13:01:02.467 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:02.465+0000 7f0a65771700 1 --2- 172.21.15.71:0/285005275 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0a600f6590 0x7f0a601067c0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T13:01:02.467 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:02.465+0000 7f0a65771700 1 -- 172.21.15.71:0/285005275 >> 172.21.15.71:0/285005275 conn(0x7f0a600f1cd0 msgr2=0x7f0a600f27c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T13:01:02.467 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:02.466+0000 7f0a65771700 1 -- 172.21.15.71:0/285005275 shutdown_connections 2024-02-20T13:01:02.467 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:02.466+0000 7f0a65771700 1 -- 172.21.15.71:0/285005275 wait complete. 2024-02-20T13:01:02.467 INFO:teuthology.orchestra.run.smithi071.stderr:dumped monmap epoch 1 2024-02-20T13:01:03.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 13:01:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T13:01:01.803861+0000 mgr.smithi064.vriyor (mgr.14156) 315 : cluster [DBG] pgmap v298: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T13:01:03.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 13:01:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T13:01:02.464715+0000 mon.smithi064 (mon.0) 288 : audit [DBG] from='client.? 172.21.15.71:0/285005275' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-20T13:01:04.241 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-20T13:01:04.242 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/daemon-base:latest-pacific shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 680ee126-cfee-11ee-95bb-87774f69a715 -- ceph mon dump -f json 2024-02-20T13:01:05.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 13:01:05 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T13:01:03.804250+0000 mgr.smithi064.vriyor (mgr.14156) 316 : cluster [DBG] pgmap v299: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T13:01:06.200 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:06.183+0000 7f7d4c862700 1 -- 172.21.15.71:0/3613917057 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7d480f5480 msgr2=0x7f7d480f58a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T13:01:06.200 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:06.183+0000 7f7d4c862700 1 --2- 172.21.15.71:0/3613917057 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7d480f5480 0x7f7d480f58a0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 rx=0x7f7d38009fb0 tx=0x7f7d38009b30).stop 2024-02-20T13:01:06.201 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:06.183+0000 7f7d4c862700 1 -- 172.21.15.71:0/3613917057 shutdown_connections 2024-02-20T13:01:06.201 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:06.183+0000 7f7d4c862700 1 --2- 172.21.15.71:0/3613917057 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7d480f5480 0x7f7d480f58a0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T13:01:06.201 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:06.183+0000 7f7d4c862700 1 -- 172.21.15.71:0/3613917057 >> 172.21.15.71:0/3613917057 conn(0x7f7d480f0be0 msgr2=0x7f7d480f3000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T13:01:06.201 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:06.183+0000 7f7d4c862700 1 -- 172.21.15.71:0/3613917057 shutdown_connections 2024-02-20T13:01:06.201 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:06.183+0000 7f7d4c862700 1 -- 172.21.15.71:0/3613917057 wait complete. 2024-02-20T13:01:06.201 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:06.184+0000 7f7d4c862700 1 Processor -- start 2024-02-20T13:01:06.201 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:06.184+0000 7f7d4c862700 1 -- start start 2024-02-20T13:01:06.201 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:06.184+0000 7f7d4c862700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7d480f5480 0x7f7d48109510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T13:01:06.201 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:06.184+0000 7f7d4c862700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d48060300 con 0x7f7d480f5480 2024-02-20T13:01:06.201 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:06.184+0000 7f7d4659c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7d480f5480 0x7f7d48109510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T13:01:06.201 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:06.184+0000 7f7d4659c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7d480f5480 0x7f7d48109510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.71:56546/0 (socket says 172.21.15.71:56546) 2024-02-20T13:01:06.201 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:06.184+0000 7f7d4659c700 1 -- 172.21.15.71:0/4094245071 learned_addr learned my addr 172.21.15.71:0/4094245071 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-02-20T13:01:06.201 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:06.184+0000 7f7d4659c700 1 -- 172.21.15.71:0/4094245071 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d38005040 con 0x7f7d480f5480 2024-02-20T13:01:06.202 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:06.185+0000 7f7d4659c700 1 --2- 172.21.15.71:0/4094245071 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7d480f5480 0x7f7d48109510 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 rx=0x7f7d380089f0 tx=0x7f7d3800b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T13:01:06.202 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:06.185+0000 7f7d36ffd700 1 -- 172.21.15.71:0/4094245071 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7d38009320 con 0x7f7d480f5480 2024-02-20T13:01:06.202 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:06.185+0000 7f7d4c862700 1 -- 172.21.15.71:0/4094245071 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d48060500 con 0x7f7d480f5480 2024-02-20T13:01:06.202 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:06.185+0000 7f7d36ffd700 1 -- 172.21.15.71:0/4094245071 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(25 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f7d38013ec0 con 0x7f7d480f5480 2024-02-20T13:01:06.202 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:06.185+0000 7f7d36ffd700 1 -- 172.21.15.71:0/4094245071 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7d38014d30 con 0x7f7d480f5480 2024-02-20T13:01:06.202 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:06.185+0000 7f7d4c862700 1 -- 172.21.15.71:0/4094245071 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d48109bc0 con 0x7f7d480f5480 2024-02-20T13:01:06.202 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:06.186+0000 7f7d36ffd700 1 -- 172.21.15.71:0/4094245071 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 37151+0+0 (secure 0 0 0) 0x7f7d380095f0 con 0x7f7d480f5480 2024-02-20T13:01:06.202 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:06.186+0000 7f7d36ffd700 1 --2- 172.21.15.71:0/4094245071 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f7d30031090 0x7f7d30033550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T13:01:06.202 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:06.186+0000 7f7d4c862700 1 -- 172.21.15.71:0/4094245071 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d480591e0 con 0x7f7d480f5480 2024-02-20T13:01:06.202 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:06.186+0000 7f7d36ffd700 1 -- 172.21.15.71:0/4094245071 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1065+0+0 (secure 0 0 0) 0x7f7d38042420 con 0x7f7d480f5480 2024-02-20T13:01:06.202 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:06.186+0000 7f7d45d9b700 1 --2- 172.21.15.71:0/4094245071 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f7d30031090 0x7f7d30033550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T13:01:06.202 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:06.186+0000 7f7d45d9b700 1 --2- 172.21.15.71:0/4094245071 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f7d30031090 0x7f7d30033550 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 rx=0x7f7d3c0096a0 tx=0x7f7d3c006b40).ready entity=mgr.14156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T13:01:06.202 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:06.191+0000 7f7d36ffd700 1 -- 172.21.15.71:0/4094245071 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+153237 (secure 0 0 0) 0x7f7d3800fe40 con 0x7f7d480f5480 2024-02-20T13:01:06.396 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:06.396+0000 7f7d4c862700 1 -- 172.21.15.71:0/4094245071 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f7d48045bb0 con 0x7f7d480f5480 2024-02-20T13:01:06.397 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:06.396+0000 7f7d36ffd700 1 -- 172.21.15.71:0/4094245071 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7f7d3800fe40 con 0x7f7d480f5480 2024-02-20T13:01:06.397 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-20T13:01:06.397 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":1,"fsid":"680ee126-cfee-11ee-95bb-87774f69a715","modified":"2024-02-20T12:49:44.699059Z","created":"2024-02-20T12:49:44.699059Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi064","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:3300","nonce":0},{"type":"v1","addr":"172.21.15.64:6789","nonce":0}]},"addr":"172.21.15.64:6789/0","public_addr":"172.21.15.64:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-20T13:01:06.399 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:06.398+0000 7f7d4c862700 1 -- 172.21.15.71:0/4094245071 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f7d30031090 msgr2=0x7f7d30033550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T13:01:06.399 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:06.398+0000 7f7d4c862700 1 --2- 172.21.15.71:0/4094245071 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f7d30031090 0x7f7d30033550 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 rx=0x7f7d3c0096a0 tx=0x7f7d3c006b40).stop 2024-02-20T13:01:06.399 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:06.398+0000 7f7d4c862700 1 -- 172.21.15.71:0/4094245071 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7d480f5480 msgr2=0x7f7d48109510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T13:01:06.399 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:06.398+0000 7f7d4c862700 1 --2- 172.21.15.71:0/4094245071 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7d480f5480 0x7f7d48109510 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 rx=0x7f7d380089f0 tx=0x7f7d3800b040).stop 2024-02-20T13:01:06.399 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:06.398+0000 7f7d4c862700 1 -- 172.21.15.71:0/4094245071 shutdown_connections 2024-02-20T13:01:06.399 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:06.398+0000 7f7d4c862700 1 --2- 172.21.15.71:0/4094245071 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f7d30031090 0x7f7d30033550 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T13:01:06.399 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:06.398+0000 7f7d4c862700 1 --2- 172.21.15.71:0/4094245071 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7d480f5480 0x7f7d48109510 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T13:01:06.399 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:06.398+0000 7f7d4c862700 1 -- 172.21.15.71:0/4094245071 >> 172.21.15.71:0/4094245071 conn(0x7f7d480f0be0 msgr2=0x7f7d480f16d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T13:01:06.399 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:06.398+0000 7f7d4c862700 1 -- 172.21.15.71:0/4094245071 shutdown_connections 2024-02-20T13:01:06.399 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:06.398+0000 7f7d4c862700 1 -- 172.21.15.71:0/4094245071 wait complete. 2024-02-20T13:01:06.400 INFO:teuthology.orchestra.run.smithi071.stderr:dumped monmap epoch 1 2024-02-20T13:01:07.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 13:01:07 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T13:01:05.804553+0000 mgr.smithi064.vriyor (mgr.14156) 317 : cluster [DBG] pgmap v300: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T13:01:07.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 13:01:07 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T13:01:06.397189+0000 mon.smithi064 (mon.0) 289 : audit [DBG] from='client.? 172.21.15.71:0/4094245071' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-20T13:01:08.086 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-20T13:01:08.087 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/daemon-base:latest-pacific shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 680ee126-cfee-11ee-95bb-87774f69a715 -- ceph mon dump -f json 2024-02-20T13:01:09.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 13:01:09 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T13:01:07.804929+0000 mgr.smithi064.vriyor (mgr.14156) 318 : cluster [DBG] pgmap v301: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T13:01:10.026 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:10.023+0000 7f7fff22e700 1 -- 172.21.15.71:0/2756486614 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7ff80f54b0 msgr2=0x7f7ff80f58d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T13:01:10.027 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:10.023+0000 7f7fff22e700 1 --2- 172.21.15.71:0/2756486614 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7ff80f54b0 0x7f7ff80f58d0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 rx=0x7f7fec009fb0 tx=0x7f7fec009b30).stop 2024-02-20T13:01:10.027 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:10.023+0000 7f7fff22e700 1 -- 172.21.15.71:0/2756486614 shutdown_connections 2024-02-20T13:01:10.027 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:10.023+0000 7f7fff22e700 1 --2- 172.21.15.71:0/2756486614 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7ff80f54b0 0x7f7ff80f58d0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T13:01:10.027 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:10.023+0000 7f7fff22e700 1 -- 172.21.15.71:0/2756486614 >> 172.21.15.71:0/2756486614 conn(0x7f7ff80f0c10 msgr2=0x7f7ff80f3030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T13:01:10.027 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:10.023+0000 7f7fff22e700 1 -- 172.21.15.71:0/2756486614 shutdown_connections 2024-02-20T13:01:10.027 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:10.023+0000 7f7fff22e700 1 -- 172.21.15.71:0/2756486614 wait complete. 2024-02-20T13:01:10.027 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:10.024+0000 7f7fff22e700 1 Processor -- start 2024-02-20T13:01:10.027 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:10.024+0000 7f7fff22e700 1 -- start start 2024-02-20T13:01:10.027 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:10.024+0000 7f7fff22e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7ff80f54b0 0x7f7ff81094c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T13:01:10.027 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:10.024+0000 7f7fff22e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ff8060300 con 0x7f7ff80f54b0 2024-02-20T13:01:10.028 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:10.024+0000 7f7ffcfca700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7ff80f54b0 0x7f7ff81094c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T13:01:10.028 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:10.024+0000 7f7ffcfca700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7ff80f54b0 0x7f7ff81094c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.71:44180/0 (socket says 172.21.15.71:44180) 2024-02-20T13:01:10.028 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:10.024+0000 7f7ffcfca700 1 -- 172.21.15.71:0/1736302676 learned_addr learned my addr 172.21.15.71:0/1736302676 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-02-20T13:01:10.028 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:10.025+0000 7f7ffcfca700 1 -- 172.21.15.71:0/1736302676 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fec005040 con 0x7f7ff80f54b0 2024-02-20T13:01:10.028 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:10.025+0000 7f7ffcfca700 1 --2- 172.21.15.71:0/1736302676 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7ff80f54b0 0x7f7ff81094c0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 rx=0x7f7fec000c00 tx=0x7f7fec00b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T13:01:10.028 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:10.025+0000 7f7ff57fa700 1 -- 172.21.15.71:0/1736302676 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7fec0095f0 con 0x7f7ff80f54b0 2024-02-20T13:01:10.028 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:10.025+0000 7f7fff22e700 1 -- 172.21.15.71:0/1736302676 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ff8060500 con 0x7f7ff80f54b0 2024-02-20T13:01:10.028 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:10.025+0000 7f7ff57fa700 1 -- 172.21.15.71:0/1736302676 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(25 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f7fec007e50 con 0x7f7ff80f54b0 2024-02-20T13:01:10.028 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:10.025+0000 7f7ff57fa700 1 -- 172.21.15.71:0/1736302676 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7fec014e70 con 0x7f7ff80f54b0 2024-02-20T13:01:10.028 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:10.025+0000 7f7fff22e700 1 -- 172.21.15.71:0/1736302676 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ff8109b70 con 0x7f7ff80f54b0 2024-02-20T13:01:10.029 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:10.026+0000 7f7ff57fa700 1 -- 172.21.15.71:0/1736302676 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 37151+0+0 (secure 0 0 0) 0x7f7fec0146b0 con 0x7f7ff80f54b0 2024-02-20T13:01:10.029 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:10.026+0000 7f7ff57fa700 1 --2- 172.21.15.71:0/1736302676 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f7fe8031090 0x7f7fe8033550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T13:01:10.029 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:10.026+0000 7f7fff22e700 1 -- 172.21.15.71:0/1736302676 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ff80591e0 con 0x7f7ff80f54b0 2024-02-20T13:01:10.029 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:10.026+0000 7f7ff57fa700 1 -- 172.21.15.71:0/1736302676 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1065+0+0 (secure 0 0 0) 0x7f7fec041d10 con 0x7f7ff80f54b0 2024-02-20T13:01:10.029 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:10.026+0000 7f7ff7fff700 1 --2- 172.21.15.71:0/1736302676 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f7fe8031090 0x7f7fe8033550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T13:01:10.030 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:10.026+0000 7f7ff7fff700 1 --2- 172.21.15.71:0/1736302676 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f7fe8031090 0x7f7fe8033550 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 rx=0x7f7fe40096a0 tx=0x7f7fe4006b40).ready entity=mgr.14156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T13:01:10.032 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:10.031+0000 7f7ff57fa700 1 -- 172.21.15.71:0/1736302676 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+153237 (secure 0 0 0) 0x7f7fec01b2c0 con 0x7f7ff80f54b0 2024-02-20T13:01:10.236 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:10.235+0000 7f7fff22e700 1 -- 172.21.15.71:0/1736302676 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f7ff8045bb0 con 0x7f7ff80f54b0 2024-02-20T13:01:10.237 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:10.235+0000 7f7ff57fa700 1 -- 172.21.15.71:0/1736302676 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7f7fec01b2c0 con 0x7f7ff80f54b0 2024-02-20T13:01:10.237 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-20T13:01:10.237 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":1,"fsid":"680ee126-cfee-11ee-95bb-87774f69a715","modified":"2024-02-20T12:49:44.699059Z","created":"2024-02-20T12:49:44.699059Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi064","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:3300","nonce":0},{"type":"v1","addr":"172.21.15.64:6789","nonce":0}]},"addr":"172.21.15.64:6789/0","public_addr":"172.21.15.64:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-20T13:01:10.239 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:10.237+0000 7f7fff22e700 1 -- 172.21.15.71:0/1736302676 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f7fe8031090 msgr2=0x7f7fe8033550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T13:01:10.239 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:10.237+0000 7f7fff22e700 1 --2- 172.21.15.71:0/1736302676 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f7fe8031090 0x7f7fe8033550 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 rx=0x7f7fe40096a0 tx=0x7f7fe4006b40).stop 2024-02-20T13:01:10.239 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:10.237+0000 7f7fff22e700 1 -- 172.21.15.71:0/1736302676 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7ff80f54b0 msgr2=0x7f7ff81094c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T13:01:10.239 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:10.237+0000 7f7fff22e700 1 --2- 172.21.15.71:0/1736302676 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7ff80f54b0 0x7f7ff81094c0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 rx=0x7f7fec000c00 tx=0x7f7fec00b040).stop 2024-02-20T13:01:10.239 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:10.237+0000 7f7fff22e700 1 -- 172.21.15.71:0/1736302676 shutdown_connections 2024-02-20T13:01:10.239 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:10.237+0000 7f7fff22e700 1 --2- 172.21.15.71:0/1736302676 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f7fe8031090 0x7f7fe8033550 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T13:01:10.239 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:10.237+0000 7f7fff22e700 1 --2- 172.21.15.71:0/1736302676 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7ff80f54b0 0x7f7ff81094c0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T13:01:10.239 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:10.237+0000 7f7fff22e700 1 -- 172.21.15.71:0/1736302676 >> 172.21.15.71:0/1736302676 conn(0x7f7ff80f0c10 msgr2=0x7f7ff80f1670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T13:01:10.239 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:10.237+0000 7f7fff22e700 1 -- 172.21.15.71:0/1736302676 shutdown_connections 2024-02-20T13:01:10.239 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:10.237+0000 7f7fff22e700 1 -- 172.21.15.71:0/1736302676 wait complete. 2024-02-20T13:01:10.240 INFO:teuthology.orchestra.run.smithi071.stderr:dumped monmap epoch 1 2024-02-20T13:01:11.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 13:01:11 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T13:01:09.805328+0000 mgr.smithi064.vriyor (mgr.14156) 319 : cluster [DBG] pgmap v302: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T13:01:11.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 13:01:11 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T13:01:10.237250+0000 mon.smithi064 (mon.0) 290 : audit [DBG] from='client.? 172.21.15.71:0/1736302676' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-20T13:01:11.929 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-20T13:01:11.930 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/daemon-base:latest-pacific shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 680ee126-cfee-11ee-95bb-87774f69a715 -- ceph mon dump -f json 2024-02-20T13:01:13.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 13:01:13 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T13:01:11.805667+0000 mgr.smithi064.vriyor (mgr.14156) 320 : cluster [DBG] pgmap v303: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T13:01:13.861 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:13.847+0000 7fcf0ba22700 1 -- 172.21.15.71:0/682090256 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcf040f54b0 msgr2=0x7fcf040f58d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T13:01:13.861 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:13.847+0000 7fcf0ba22700 1 --2- 172.21.15.71:0/682090256 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcf040f54b0 0x7fcf040f58d0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 rx=0x7fcef4009fb0 tx=0x7fcef4009b30).stop 2024-02-20T13:01:13.861 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:13.847+0000 7fcf0ba22700 1 -- 172.21.15.71:0/682090256 shutdown_connections 2024-02-20T13:01:13.861 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:13.847+0000 7fcf0ba22700 1 --2- 172.21.15.71:0/682090256 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcf040f54b0 0x7fcf040f58d0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T13:01:13.861 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:13.847+0000 7fcf0ba22700 1 -- 172.21.15.71:0/682090256 >> 172.21.15.71:0/682090256 conn(0x7fcf040f0c10 msgr2=0x7fcf040f3030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T13:01:13.861 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:13.847+0000 7fcf0ba22700 1 -- 172.21.15.71:0/682090256 shutdown_connections 2024-02-20T13:01:13.862 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:13.847+0000 7fcf0ba22700 1 -- 172.21.15.71:0/682090256 wait complete. 2024-02-20T13:01:13.862 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:13.848+0000 7fcf0ba22700 1 Processor -- start 2024-02-20T13:01:13.862 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:13.848+0000 7fcf0ba22700 1 -- start start 2024-02-20T13:01:13.862 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:13.848+0000 7fcf0ba22700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcf040f54b0 0x7fcf04105700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T13:01:13.862 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:13.848+0000 7fcf0ba22700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf04105bf0 con 0x7fcf040f54b0 2024-02-20T13:01:13.862 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:13.848+0000 7fcf097be700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcf040f54b0 0x7fcf04105700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T13:01:13.862 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:13.848+0000 7fcf097be700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcf040f54b0 0x7fcf04105700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.71:44190/0 (socket says 172.21.15.71:44190) 2024-02-20T13:01:13.862 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:13.848+0000 7fcf097be700 1 -- 172.21.15.71:0/824970303 learned_addr learned my addr 172.21.15.71:0/824970303 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-02-20T13:01:13.862 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:13.849+0000 7fcf097be700 1 -- 172.21.15.71:0/824970303 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcef4005040 con 0x7fcf040f54b0 2024-02-20T13:01:13.862 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:13.849+0000 7fcf097be700 1 --2- 172.21.15.71:0/824970303 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcf040f54b0 0x7fcf04105700 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 rx=0x7fcef40089f0 tx=0x7fcef400b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T13:01:13.862 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:13.849+0000 7fcef9ffb700 1 -- 172.21.15.71:0/824970303 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fcef4009320 con 0x7fcf040f54b0 2024-02-20T13:01:13.862 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:13.849+0000 7fcef9ffb700 1 -- 172.21.15.71:0/824970303 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(25 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7fcef4013ec0 con 0x7fcf040f54b0 2024-02-20T13:01:13.862 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:13.849+0000 7fcf0ba22700 1 -- 172.21.15.71:0/824970303 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf04105df0 con 0x7fcf040f54b0 2024-02-20T13:01:13.862 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:13.849+0000 7fcef9ffb700 1 -- 172.21.15.71:0/824970303 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fcef4014d30 con 0x7fcf040f54b0 2024-02-20T13:01:13.862 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:13.849+0000 7fcf0ba22700 1 -- 172.21.15.71:0/824970303 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf04106280 con 0x7fcf040f54b0 2024-02-20T13:01:13.863 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:13.850+0000 7fcef9ffb700 1 -- 172.21.15.71:0/824970303 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 37151+0+0 (secure 0 0 0) 0x7fcef40095f0 con 0x7fcf040f54b0 2024-02-20T13:01:13.863 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:13.850+0000 7fcf0ba22700 1 -- 172.21.15.71:0/824970303 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf04045bb0 con 0x7fcf040f54b0 2024-02-20T13:01:13.863 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:13.850+0000 7fcef9ffb700 1 --2- 172.21.15.71:0/824970303 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fcef0031090 0x7fcef0033550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T13:01:13.863 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:13.850+0000 7fcef9ffb700 1 -- 172.21.15.71:0/824970303 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1065+0+0 (secure 0 0 0) 0x7fcef4042460 con 0x7fcf040f54b0 2024-02-20T13:01:13.863 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:13.850+0000 7fcf08fbd700 1 --2- 172.21.15.71:0/824970303 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fcef0031090 0x7fcef0033550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T13:01:13.863 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:13.851+0000 7fcf08fbd700 1 --2- 172.21.15.71:0/824970303 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fcef0031090 0x7fcef0033550 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 rx=0x7fcf000096a0 tx=0x7fcf00006b40).ready entity=mgr.14156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T13:01:13.863 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:13.855+0000 7fcef9ffb700 1 -- 172.21.15.71:0/824970303 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+153237 (secure 0 0 0) 0x7fcef401d7f0 con 0x7fcf040f54b0 2024-02-20T13:01:14.062 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:14.060+0000 7fcf0ba22700 1 -- 172.21.15.71:0/824970303 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fcf04107130 con 0x7fcf040f54b0 2024-02-20T13:01:14.062 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:14.060+0000 7fcef9ffb700 1 -- 172.21.15.71:0/824970303 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7fcef401d7f0 con 0x7fcf040f54b0 2024-02-20T13:01:14.062 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-20T13:01:14.063 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":1,"fsid":"680ee126-cfee-11ee-95bb-87774f69a715","modified":"2024-02-20T12:49:44.699059Z","created":"2024-02-20T12:49:44.699059Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi064","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:3300","nonce":0},{"type":"v1","addr":"172.21.15.64:6789","nonce":0}]},"addr":"172.21.15.64:6789/0","public_addr":"172.21.15.64:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-20T13:01:14.064 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:14.062+0000 7fcf0ba22700 1 -- 172.21.15.71:0/824970303 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fcef0031090 msgr2=0x7fcef0033550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T13:01:14.064 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:14.062+0000 7fcf0ba22700 1 --2- 172.21.15.71:0/824970303 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fcef0031090 0x7fcef0033550 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 rx=0x7fcf000096a0 tx=0x7fcf00006b40).stop 2024-02-20T13:01:14.064 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:14.062+0000 7fcf0ba22700 1 -- 172.21.15.71:0/824970303 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcf040f54b0 msgr2=0x7fcf04105700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T13:01:14.064 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:14.062+0000 7fcf0ba22700 1 --2- 172.21.15.71:0/824970303 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcf040f54b0 0x7fcf04105700 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 rx=0x7fcef40089f0 tx=0x7fcef400b040).stop 2024-02-20T13:01:14.064 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:14.062+0000 7fcf0ba22700 1 -- 172.21.15.71:0/824970303 shutdown_connections 2024-02-20T13:01:14.064 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:14.062+0000 7fcf0ba22700 1 --2- 172.21.15.71:0/824970303 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fcef0031090 0x7fcef0033550 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T13:01:14.064 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:14.062+0000 7fcf0ba22700 1 --2- 172.21.15.71:0/824970303 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcf040f54b0 0x7fcf04105700 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T13:01:14.064 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:14.062+0000 7fcf0ba22700 1 -- 172.21.15.71:0/824970303 >> 172.21.15.71:0/824970303 conn(0x7fcf040f0c10 msgr2=0x7fcf040f1700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T13:01:14.064 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:14.062+0000 7fcf0ba22700 1 -- 172.21.15.71:0/824970303 shutdown_connections 2024-02-20T13:01:14.065 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:14.062+0000 7fcf0ba22700 1 -- 172.21.15.71:0/824970303 wait complete. 2024-02-20T13:01:14.065 INFO:teuthology.orchestra.run.smithi071.stderr:dumped monmap epoch 1 2024-02-20T13:01:15.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 13:01:15 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T13:01:13.805989+0000 mgr.smithi064.vriyor (mgr.14156) 321 : cluster [DBG] pgmap v304: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T13:01:15.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 13:01:15 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T13:01:14.062412+0000 mon.smithi064 (mon.0) 291 : audit [DBG] from='client.? 172.21.15.71:0/824970303' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-20T13:01:15.707 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-20T13:01:15.708 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/daemon-base:latest-pacific shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 680ee126-cfee-11ee-95bb-87774f69a715 -- ceph mon dump -f json 2024-02-20T13:01:17.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 13:01:17 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T13:01:15.806340+0000 mgr.smithi064.vriyor (mgr.14156) 322 : cluster [DBG] pgmap v305: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T13:01:17.658 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:17.655+0000 7f05f24d3700 1 -- 172.21.15.71:0/1047971349 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05ec0f5480 msgr2=0x7f05ec0f58a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T13:01:17.658 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:17.655+0000 7f05f24d3700 1 --2- 172.21.15.71:0/1047971349 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05ec0f5480 0x7f05ec0f58a0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 rx=0x7f05dc009fb0 tx=0x7f05dc009b30).stop 2024-02-20T13:01:17.658 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:17.655+0000 7f05f24d3700 1 -- 172.21.15.71:0/1047971349 shutdown_connections 2024-02-20T13:01:17.658 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:17.655+0000 7f05f24d3700 1 --2- 172.21.15.71:0/1047971349 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05ec0f5480 0x7f05ec0f58a0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T13:01:17.658 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:17.655+0000 7f05f24d3700 1 -- 172.21.15.71:0/1047971349 >> 172.21.15.71:0/1047971349 conn(0x7f05ec0f0be0 msgr2=0x7f05ec0f3000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T13:01:17.659 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:17.655+0000 7f05f24d3700 1 -- 172.21.15.71:0/1047971349 shutdown_connections 2024-02-20T13:01:17.659 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:17.655+0000 7f05f24d3700 1 -- 172.21.15.71:0/1047971349 wait complete. 2024-02-20T13:01:17.659 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:17.656+0000 7f05f24d3700 1 Processor -- start 2024-02-20T13:01:17.659 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:17.656+0000 7f05f24d3700 1 -- start start 2024-02-20T13:01:17.659 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:17.656+0000 7f05f24d3700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05ec0f5480 0x7f05ec10b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T13:01:17.659 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:17.656+0000 7f05f24d3700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05ec107300 con 0x7f05ec0f5480 2024-02-20T13:01:17.659 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:17.656+0000 7f05ebfff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05ec0f5480 0x7f05ec10b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T13:01:17.659 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:17.656+0000 7f05ebfff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05ec0f5480 0x7f05ec10b910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.71:46770/0 (socket says 172.21.15.71:46770) 2024-02-20T13:01:17.659 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:17.656+0000 7f05ebfff700 1 -- 172.21.15.71:0/2649638571 learned_addr learned my addr 172.21.15.71:0/2649638571 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-02-20T13:01:17.659 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:17.656+0000 7f05ebfff700 1 -- 172.21.15.71:0/2649638571 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05dc005040 con 0x7f05ec0f5480 2024-02-20T13:01:17.659 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:17.656+0000 7f05ebfff700 1 --2- 172.21.15.71:0/2649638571 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05ec0f5480 0x7f05ec10b910 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 rx=0x7f05dc0089f0 tx=0x7f05dc00b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T13:01:17.659 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:17.657+0000 7f05e8ff9700 1 -- 172.21.15.71:0/2649638571 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f05dc009320 con 0x7f05ec0f5480 2024-02-20T13:01:17.659 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:17.657+0000 7f05e8ff9700 1 -- 172.21.15.71:0/2649638571 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(25 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f05dc013ec0 con 0x7f05ec0f5480 2024-02-20T13:01:17.659 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:17.657+0000 7f05f24d3700 1 -- 172.21.15.71:0/2649638571 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05ec060300 con 0x7f05ec0f5480 2024-02-20T13:01:17.660 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:17.657+0000 7f05e8ff9700 1 -- 172.21.15.71:0/2649638571 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f05dc014d30 con 0x7f05ec0f5480 2024-02-20T13:01:17.660 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:17.657+0000 7f05f24d3700 1 -- 172.21.15.71:0/2649638571 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05ec10be00 con 0x7f05ec0f5480 2024-02-20T13:01:17.660 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:17.657+0000 7f05e8ff9700 1 -- 172.21.15.71:0/2649638571 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 37151+0+0 (secure 0 0 0) 0x7f05dc0095f0 con 0x7f05ec0f5480 2024-02-20T13:01:17.660 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:17.658+0000 7f05f24d3700 1 -- 172.21.15.71:0/2649638571 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05ec045bb0 con 0x7f05ec0f5480 2024-02-20T13:01:17.661 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:17.658+0000 7f05e8ff9700 1 --2- 172.21.15.71:0/2649638571 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f05d4031090 0x7f05d4033550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T13:01:17.661 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:17.658+0000 7f05e8ff9700 1 -- 172.21.15.71:0/2649638571 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1065+0+0 (secure 0 0 0) 0x7f05dc0424a0 con 0x7f05ec0f5480 2024-02-20T13:01:17.661 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:17.658+0000 7f05eb7fe700 1 --2- 172.21.15.71:0/2649638571 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f05d4031090 0x7f05d4033550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T13:01:17.661 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:17.658+0000 7f05eb7fe700 1 --2- 172.21.15.71:0/2649638571 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f05d4031090 0x7f05d4033550 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 rx=0x7f05e00096a0 tx=0x7f05e0006b40).ready entity=mgr.14156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T13:01:17.664 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:17.663+0000 7f05e8ff9700 1 -- 172.21.15.71:0/2649638571 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+153237 (secure 0 0 0) 0x7f05dc0268c0 con 0x7f05ec0f5480 2024-02-20T13:01:17.871 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:17.869+0000 7f05f24d3700 1 -- 172.21.15.71:0/2649638571 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f05ec0faf70 con 0x7f05ec0f5480 2024-02-20T13:01:17.871 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:17.871+0000 7f05e8ff9700 1 -- 172.21.15.71:0/2649638571 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7f05dc01d7f0 con 0x7f05ec0f5480 2024-02-20T13:01:17.872 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-20T13:01:17.872 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":1,"fsid":"680ee126-cfee-11ee-95bb-87774f69a715","modified":"2024-02-20T12:49:44.699059Z","created":"2024-02-20T12:49:44.699059Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi064","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:3300","nonce":0},{"type":"v1","addr":"172.21.15.64:6789","nonce":0}]},"addr":"172.21.15.64:6789/0","public_addr":"172.21.15.64:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-20T13:01:17.873 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:17.871+0000 7f05f24d3700 1 -- 172.21.15.71:0/2649638571 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f05d4031090 msgr2=0x7f05d4033550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T13:01:17.873 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:17.871+0000 7f05f24d3700 1 --2- 172.21.15.71:0/2649638571 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f05d4031090 0x7f05d4033550 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 rx=0x7f05e00096a0 tx=0x7f05e0006b40).stop 2024-02-20T13:01:17.873 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:17.871+0000 7f05f24d3700 1 -- 172.21.15.71:0/2649638571 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05ec0f5480 msgr2=0x7f05ec10b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T13:01:17.873 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:17.871+0000 7f05f24d3700 1 --2- 172.21.15.71:0/2649638571 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05ec0f5480 0x7f05ec10b910 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 rx=0x7f05dc0089f0 tx=0x7f05dc00b040).stop 2024-02-20T13:01:17.873 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:17.872+0000 7f05f24d3700 1 -- 172.21.15.71:0/2649638571 shutdown_connections 2024-02-20T13:01:17.873 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:17.872+0000 7f05f24d3700 1 --2- 172.21.15.71:0/2649638571 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f05d4031090 0x7f05d4033550 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T13:01:17.873 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:17.872+0000 7f05f24d3700 1 --2- 172.21.15.71:0/2649638571 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05ec0f5480 0x7f05ec10b910 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T13:01:17.874 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:17.872+0000 7f05f24d3700 1 -- 172.21.15.71:0/2649638571 >> 172.21.15.71:0/2649638571 conn(0x7f05ec0f0be0 msgr2=0x7f05ec0f16d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T13:01:17.874 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:17.872+0000 7f05f24d3700 1 -- 172.21.15.71:0/2649638571 shutdown_connections 2024-02-20T13:01:17.874 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:17.872+0000 7f05f24d3700 1 -- 172.21.15.71:0/2649638571 wait complete. 2024-02-20T13:01:17.874 INFO:teuthology.orchestra.run.smithi071.stderr:dumped monmap epoch 1 2024-02-20T13:01:18.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 13:01:18 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T13:01:17.871785+0000 mon.smithi064 (mon.0) 292 : audit [DBG] from='client.? 172.21.15.71:0/2649638571' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-20T13:01:19.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 13:01:19 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T13:01:17.806699+0000 mgr.smithi064.vriyor (mgr.14156) 323 : cluster [DBG] pgmap v306: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T13:01:19.537 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-20T13:01:19.538 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/daemon-base:latest-pacific shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 680ee126-cfee-11ee-95bb-87774f69a715 -- ceph mon dump -f json 2024-02-20T13:01:21.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 13:01:21 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T13:01:19.807076+0000 mgr.smithi064.vriyor (mgr.14156) 324 : cluster [DBG] pgmap v307: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T13:01:21.524 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:21.504+0000 7fe28467a700 1 -- 172.21.15.71:0/3177800962 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe27c0f6590 msgr2=0x7fe27c0f69b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T13:01:21.524 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:21.504+0000 7fe28467a700 1 --2- 172.21.15.71:0/3177800962 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe27c0f6590 0x7fe27c0f69b0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 rx=0x7fe278004760 tx=0x7fe278009ad0).stop 2024-02-20T13:01:21.524 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:21.504+0000 7fe28467a700 1 -- 172.21.15.71:0/3177800962 shutdown_connections 2024-02-20T13:01:21.524 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:21.504+0000 7fe28467a700 1 --2- 172.21.15.71:0/3177800962 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe27c0f6590 0x7fe27c0f69b0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T13:01:21.524 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:21.504+0000 7fe28467a700 1 -- 172.21.15.71:0/3177800962 >> 172.21.15.71:0/3177800962 conn(0x7fe27c0f1cd0 msgr2=0x7fe27c0f4110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T13:01:21.524 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:21.504+0000 7fe28467a700 1 -- 172.21.15.71:0/3177800962 shutdown_connections 2024-02-20T13:01:21.524 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:21.504+0000 7fe28467a700 1 -- 172.21.15.71:0/3177800962 wait complete. 2024-02-20T13:01:21.524 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:21.505+0000 7fe28467a700 1 Processor -- start 2024-02-20T13:01:21.524 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:21.505+0000 7fe28467a700 1 -- start start 2024-02-20T13:01:21.524 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:21.505+0000 7fe28467a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe27c0f6590 0x7fe27c1067c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T13:01:21.525 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:21.505+0000 7fe28467a700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe27c106cb0 con 0x7fe27c0f6590 2024-02-20T13:01:21.525 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:21.506+0000 7fe282416700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe27c0f6590 0x7fe27c1067c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T13:01:21.525 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:21.506+0000 7fe282416700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe27c0f6590 0x7fe27c1067c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.71:46784/0 (socket says 172.21.15.71:46784) 2024-02-20T13:01:21.525 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:21.506+0000 7fe282416700 1 -- 172.21.15.71:0/1067042910 learned_addr learned my addr 172.21.15.71:0/1067042910 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-02-20T13:01:21.525 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:21.506+0000 7fe282416700 1 -- 172.21.15.71:0/1067042910 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe278009d70 con 0x7fe27c0f6590 2024-02-20T13:01:21.525 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:21.506+0000 7fe282416700 1 --2- 172.21.15.71:0/1067042910 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe27c0f6590 0x7fe27c1067c0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 rx=0x7fe278004170 tx=0x7fe278009ad0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T13:01:21.525 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:21.506+0000 7fe272ffd700 1 -- 172.21.15.71:0/1067042910 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe27800e040 con 0x7fe27c0f6590 2024-02-20T13:01:21.525 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:21.506+0000 7fe28467a700 1 -- 172.21.15.71:0/1067042910 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe27c106eb0 con 0x7fe27c0f6590 2024-02-20T13:01:21.525 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:21.506+0000 7fe272ffd700 1 -- 172.21.15.71:0/1067042910 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(25 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7fe278008ef0 con 0x7fe27c0f6590 2024-02-20T13:01:21.525 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:21.506+0000 7fe272ffd700 1 -- 172.21.15.71:0/1067042910 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe278014c90 con 0x7fe27c0f6590 2024-02-20T13:01:21.525 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:21.506+0000 7fe28467a700 1 -- 172.21.15.71:0/1067042910 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe27c107340 con 0x7fe27c0f6590 2024-02-20T13:01:21.525 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:21.507+0000 7fe272ffd700 1 -- 172.21.15.71:0/1067042910 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 37151+0+0 (secure 0 0 0) 0x7fe278014420 con 0x7fe27c0f6590 2024-02-20T13:01:21.525 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:21.507+0000 7fe28467a700 1 -- 172.21.15.71:0/1067042910 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe27c045bb0 con 0x7fe27c0f6590 2024-02-20T13:01:21.525 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:21.507+0000 7fe272ffd700 1 --2- 172.21.15.71:0/1067042910 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fe268031090 0x7fe268033550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T13:01:21.526 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:21.507+0000 7fe272ffd700 1 -- 172.21.15.71:0/1067042910 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1065+0+0 (secure 0 0 0) 0x7fe278042210 con 0x7fe27c0f6590 2024-02-20T13:01:21.526 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:21.507+0000 7fe281c15700 1 --2- 172.21.15.71:0/1067042910 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fe268031090 0x7fe268033550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T13:01:21.526 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:21.508+0000 7fe281c15700 1 --2- 172.21.15.71:0/1067042910 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fe268031090 0x7fe268033550 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 rx=0x7fe26c0096a0 tx=0x7fe26c006b40).ready entity=mgr.14156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T13:01:21.526 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:21.512+0000 7fe272ffd700 1 -- 172.21.15.71:0/1067042910 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+153237 (secure 0 0 0) 0x7fe278026300 con 0x7fe27c0f6590 2024-02-20T13:01:21.719 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:21.718+0000 7fe28467a700 1 -- 172.21.15.71:0/1067042910 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fe27c108170 con 0x7fe27c0f6590 2024-02-20T13:01:21.720 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:21.718+0000 7fe272ffd700 1 -- 172.21.15.71:0/1067042910 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7fe278012b20 con 0x7fe27c0f6590 2024-02-20T13:01:21.720 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-20T13:01:21.720 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":1,"fsid":"680ee126-cfee-11ee-95bb-87774f69a715","modified":"2024-02-20T12:49:44.699059Z","created":"2024-02-20T12:49:44.699059Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi064","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:3300","nonce":0},{"type":"v1","addr":"172.21.15.64:6789","nonce":0}]},"addr":"172.21.15.64:6789/0","public_addr":"172.21.15.64:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-20T13:01:21.722 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:21.720+0000 7fe28467a700 1 -- 172.21.15.71:0/1067042910 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fe268031090 msgr2=0x7fe268033550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T13:01:21.722 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:21.720+0000 7fe28467a700 1 --2- 172.21.15.71:0/1067042910 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fe268031090 0x7fe268033550 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 rx=0x7fe26c0096a0 tx=0x7fe26c006b40).stop 2024-02-20T13:01:21.722 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:21.720+0000 7fe28467a700 1 -- 172.21.15.71:0/1067042910 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe27c0f6590 msgr2=0x7fe27c1067c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T13:01:21.722 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:21.720+0000 7fe28467a700 1 --2- 172.21.15.71:0/1067042910 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe27c0f6590 0x7fe27c1067c0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 rx=0x7fe278004170 tx=0x7fe278009ad0).stop 2024-02-20T13:01:21.722 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:21.720+0000 7fe28467a700 1 -- 172.21.15.71:0/1067042910 shutdown_connections 2024-02-20T13:01:21.722 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:21.720+0000 7fe28467a700 1 --2- 172.21.15.71:0/1067042910 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fe268031090 0x7fe268033550 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T13:01:21.722 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:21.720+0000 7fe28467a700 1 --2- 172.21.15.71:0/1067042910 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe27c0f6590 0x7fe27c1067c0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T13:01:21.722 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:21.720+0000 7fe28467a700 1 -- 172.21.15.71:0/1067042910 >> 172.21.15.71:0/1067042910 conn(0x7fe27c0f1cd0 msgr2=0x7fe27c0f27c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T13:01:21.722 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:21.720+0000 7fe28467a700 1 -- 172.21.15.71:0/1067042910 shutdown_connections 2024-02-20T13:01:21.722 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:21.720+0000 7fe28467a700 1 -- 172.21.15.71:0/1067042910 wait complete. 2024-02-20T13:01:21.723 INFO:teuthology.orchestra.run.smithi071.stderr:dumped monmap epoch 1 2024-02-20T13:01:22.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 13:01:22 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T13:01:21.720248+0000 mon.smithi064 (mon.0) 293 : audit [DBG] from='client.? 172.21.15.71:0/1067042910' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-20T13:01:23.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 13:01:23 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T13:01:21.807359+0000 mgr.smithi064.vriyor (mgr.14156) 325 : cluster [DBG] pgmap v308: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T13:01:23.371 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-20T13:01:23.372 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/daemon-base:latest-pacific shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 680ee126-cfee-11ee-95bb-87774f69a715 -- ceph mon dump -f json 2024-02-20T13:01:25.196 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:25.176+0000 7f796f2db700 1 -- 172.21.15.71:0/2768594378 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f79680f6590 msgr2=0x7f79680f69b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T13:01:25.196 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:25.176+0000 7f796f2db700 1 --2- 172.21.15.71:0/2768594378 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f79680f6590 0x7f79680f69b0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 rx=0x7f7958004760 tx=0x7f7958009ad0).stop 2024-02-20T13:01:25.196 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:25.176+0000 7f796f2db700 1 -- 172.21.15.71:0/2768594378 shutdown_connections 2024-02-20T13:01:25.196 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:25.176+0000 7f796f2db700 1 --2- 172.21.15.71:0/2768594378 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f79680f6590 0x7f79680f69b0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T13:01:25.196 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:25.176+0000 7f796f2db700 1 -- 172.21.15.71:0/2768594378 >> 172.21.15.71:0/2768594378 conn(0x7f79680f1cd0 msgr2=0x7f79680f4110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T13:01:25.196 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:25.176+0000 7f796f2db700 1 -- 172.21.15.71:0/2768594378 shutdown_connections 2024-02-20T13:01:25.196 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:25.176+0000 7f796f2db700 1 -- 172.21.15.71:0/2768594378 wait complete. 2024-02-20T13:01:25.196 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:25.177+0000 7f796f2db700 1 Processor -- start 2024-02-20T13:01:25.196 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:25.177+0000 7f796f2db700 1 -- start start 2024-02-20T13:01:25.196 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:25.177+0000 7f796f2db700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f79680f6590 0x7f79681067c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T13:01:25.196 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:25.177+0000 7f796f2db700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7968106cb0 con 0x7f79680f6590 2024-02-20T13:01:25.196 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:25.177+0000 7f796d077700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f79680f6590 0x7f79681067c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T13:01:25.197 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:25.177+0000 7f796d077700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f79680f6590 0x7f79681067c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.71:46802/0 (socket says 172.21.15.71:46802) 2024-02-20T13:01:25.197 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:25.177+0000 7f796d077700 1 -- 172.21.15.71:0/3231612791 learned_addr learned my addr 172.21.15.71:0/3231612791 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-02-20T13:01:25.197 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:25.178+0000 7f796d077700 1 -- 172.21.15.71:0/3231612791 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7958009d70 con 0x7f79680f6590 2024-02-20T13:01:25.197 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:25.178+0000 7f796d077700 1 --2- 172.21.15.71:0/3231612791 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f79680f6590 0x7f79681067c0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 rx=0x7f7958004170 tx=0x7f7958009ad0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T13:01:25.197 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:25.178+0000 7f795dffb700 1 -- 172.21.15.71:0/3231612791 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f795800e040 con 0x7f79680f6590 2024-02-20T13:01:25.197 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:25.178+0000 7f796f2db700 1 -- 172.21.15.71:0/3231612791 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7968106eb0 con 0x7f79680f6590 2024-02-20T13:01:25.197 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:25.178+0000 7f795dffb700 1 -- 172.21.15.71:0/3231612791 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(25 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f7958008ef0 con 0x7f79680f6590 2024-02-20T13:01:25.197 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:25.178+0000 7f795dffb700 1 -- 172.21.15.71:0/3231612791 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7958014c90 con 0x7f79680f6590 2024-02-20T13:01:25.197 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:25.178+0000 7f796f2db700 1 -- 172.21.15.71:0/3231612791 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7968107340 con 0x7f79680f6590 2024-02-20T13:01:25.197 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:25.179+0000 7f795dffb700 1 -- 172.21.15.71:0/3231612791 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 37151+0+0 (secure 0 0 0) 0x7f7958014420 con 0x7f79680f6590 2024-02-20T13:01:25.197 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:25.179+0000 7f796f2db700 1 -- 172.21.15.71:0/3231612791 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7968045bb0 con 0x7f79680f6590 2024-02-20T13:01:25.197 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:25.179+0000 7f795dffb700 1 --2- 172.21.15.71:0/3231612791 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f79540310e0 0x7f79540335a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T13:01:25.197 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:25.179+0000 7f795dffb700 1 -- 172.21.15.71:0/3231612791 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1065+0+0 (secure 0 0 0) 0x7f79580422d0 con 0x7f79680f6590 2024-02-20T13:01:25.197 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:25.179+0000 7f796c876700 1 --2- 172.21.15.71:0/3231612791 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f79540310e0 0x7f79540335a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T13:01:25.197 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:25.180+0000 7f796c876700 1 --2- 172.21.15.71:0/3231612791 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f79540310e0 0x7f79540335a0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 rx=0x7f79640096a0 tx=0x7f7964006b40).ready entity=mgr.14156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T13:01:25.197 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:25.184+0000 7f795dffb700 1 -- 172.21.15.71:0/3231612791 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+153237 (secure 0 0 0) 0x7f7958026300 con 0x7f79680f6590 2024-02-20T13:01:25.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 13:01:25 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T13:01:23.807652+0000 mgr.smithi064.vriyor (mgr.14156) 326 : cluster [DBG] pgmap v309: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T13:01:25.394 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:25.392+0000 7f796f2db700 1 -- 172.21.15.71:0/3231612791 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f7968108190 con 0x7f79680f6590 2024-02-20T13:01:25.394 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:25.393+0000 7f795dffb700 1 -- 172.21.15.71:0/3231612791 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7f7958012b20 con 0x7f79680f6590 2024-02-20T13:01:25.395 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-20T13:01:25.395 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":1,"fsid":"680ee126-cfee-11ee-95bb-87774f69a715","modified":"2024-02-20T12:49:44.699059Z","created":"2024-02-20T12:49:44.699059Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi064","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:3300","nonce":0},{"type":"v1","addr":"172.21.15.64:6789","nonce":0}]},"addr":"172.21.15.64:6789/0","public_addr":"172.21.15.64:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-20T13:01:25.396 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:25.394+0000 7f796f2db700 1 -- 172.21.15.71:0/3231612791 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f79540310e0 msgr2=0x7f79540335a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T13:01:25.396 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:25.394+0000 7f796f2db700 1 --2- 172.21.15.71:0/3231612791 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f79540310e0 0x7f79540335a0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 rx=0x7f79640096a0 tx=0x7f7964006b40).stop 2024-02-20T13:01:25.396 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:25.394+0000 7f796f2db700 1 -- 172.21.15.71:0/3231612791 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f79680f6590 msgr2=0x7f79681067c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T13:01:25.396 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:25.394+0000 7f796f2db700 1 --2- 172.21.15.71:0/3231612791 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f79680f6590 0x7f79681067c0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 rx=0x7f7958004170 tx=0x7f7958009ad0).stop 2024-02-20T13:01:25.396 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:25.395+0000 7f796f2db700 1 -- 172.21.15.71:0/3231612791 shutdown_connections 2024-02-20T13:01:25.396 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:25.395+0000 7f796f2db700 1 --2- 172.21.15.71:0/3231612791 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f79540310e0 0x7f79540335a0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T13:01:25.396 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:25.395+0000 7f796f2db700 1 --2- 172.21.15.71:0/3231612791 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f79680f6590 0x7f79681067c0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T13:01:25.397 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:25.395+0000 7f796f2db700 1 -- 172.21.15.71:0/3231612791 >> 172.21.15.71:0/3231612791 conn(0x7f79680f1cd0 msgr2=0x7f79680f27c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T13:01:25.397 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:25.395+0000 7f796f2db700 1 -- 172.21.15.71:0/3231612791 shutdown_connections 2024-02-20T13:01:25.397 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:25.395+0000 7f796f2db700 1 -- 172.21.15.71:0/3231612791 wait complete. 2024-02-20T13:01:25.397 INFO:teuthology.orchestra.run.smithi071.stderr:dumped monmap epoch 1 2024-02-20T13:01:26.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 13:01:26 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T13:01:25.394883+0000 mon.smithi064 (mon.0) 294 : audit [DBG] from='client.? 172.21.15.71:0/3231612791' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-20T13:01:27.291 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-20T13:01:27.291 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/daemon-base:latest-pacific shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 680ee126-cfee-11ee-95bb-87774f69a715 -- ceph mon dump -f json 2024-02-20T13:01:27.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 13:01:27 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T13:01:25.807900+0000 mgr.smithi064.vriyor (mgr.14156) 327 : cluster [DBG] pgmap v310: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T13:01:29.226 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:29.221+0000 7fb28ceda700 1 -- 172.21.15.71:0/1735857242 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb2880f4c00 msgr2=0x7fb2880f5020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T13:01:29.227 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:29.221+0000 7fb28ceda700 1 --2- 172.21.15.71:0/1735857242 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb2880f4c00 0x7fb2880f5020 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 rx=0x7fb270009fb0 tx=0x7fb270009b30).stop 2024-02-20T13:01:29.227 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:29.221+0000 7fb28ceda700 1 -- 172.21.15.71:0/1735857242 shutdown_connections 2024-02-20T13:01:29.227 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:29.221+0000 7fb28ceda700 1 --2- 172.21.15.71:0/1735857242 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb2880f4c00 0x7fb2880f5020 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T13:01:29.227 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:29.221+0000 7fb28ceda700 1 -- 172.21.15.71:0/1735857242 >> 172.21.15.71:0/1735857242 conn(0x7fb2880f0360 msgr2=0x7fb2880f2780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T13:01:29.227 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:29.221+0000 7fb28ceda700 1 -- 172.21.15.71:0/1735857242 shutdown_connections 2024-02-20T13:01:29.227 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:29.221+0000 7fb28ceda700 1 -- 172.21.15.71:0/1735857242 wait complete. 2024-02-20T13:01:29.227 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:29.222+0000 7fb28ceda700 1 Processor -- start 2024-02-20T13:01:29.227 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:29.222+0000 7fb28ceda700 1 -- start start 2024-02-20T13:01:29.227 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:29.222+0000 7fb28ceda700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb2880f4c00 0x7fb288109500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T13:01:29.227 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:29.222+0000 7fb28ceda700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2880f19f0 con 0x7fb2880f4c00 2024-02-20T13:01:29.227 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:29.222+0000 7fb28659c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb2880f4c00 0x7fb288109500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T13:01:29.228 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:29.222+0000 7fb28659c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb2880f4c00 0x7fb288109500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.71:38058/0 (socket says 172.21.15.71:38058) 2024-02-20T13:01:29.228 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:29.222+0000 7fb28659c700 1 -- 172.21.15.71:0/2221732708 learned_addr learned my addr 172.21.15.71:0/2221732708 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-02-20T13:01:29.228 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:29.223+0000 7fb28659c700 1 -- 172.21.15.71:0/2221732708 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb270005040 con 0x7fb2880f4c00 2024-02-20T13:01:29.228 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:29.223+0000 7fb28659c700 1 --2- 172.21.15.71:0/2221732708 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb2880f4c00 0x7fb288109500 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 rx=0x7fb2700089f0 tx=0x7fb27000b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T13:01:29.228 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:29.223+0000 7fb27effd700 1 -- 172.21.15.71:0/2221732708 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb270009320 con 0x7fb2880f4c00 2024-02-20T13:01:29.228 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:29.223+0000 7fb28ceda700 1 -- 172.21.15.71:0/2221732708 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2880f1bf0 con 0x7fb2880f4c00 2024-02-20T13:01:29.228 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:29.223+0000 7fb27effd700 1 -- 172.21.15.71:0/2221732708 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(25 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7fb270013ec0 con 0x7fb2880f4c00 2024-02-20T13:01:29.228 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:29.223+0000 7fb27effd700 1 -- 172.21.15.71:0/2221732708 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb270014d30 con 0x7fb2880f4c00 2024-02-20T13:01:29.228 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:29.223+0000 7fb28ceda700 1 -- 172.21.15.71:0/2221732708 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb288109bb0 con 0x7fb2880f4c00 2024-02-20T13:01:29.228 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:29.224+0000 7fb27effd700 1 -- 172.21.15.71:0/2221732708 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 37151+0+0 (secure 0 0 0) 0x7fb2700095f0 con 0x7fb2880f4c00 2024-02-20T13:01:29.228 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:29.224+0000 7fb28ceda700 1 -- 172.21.15.71:0/2221732708 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2880591e0 con 0x7fb2880f4c00 2024-02-20T13:01:29.228 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:29.224+0000 7fb27effd700 1 --2- 172.21.15.71:0/2221732708 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fb274031090 0x7fb274033550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T13:01:29.228 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:29.224+0000 7fb27effd700 1 -- 172.21.15.71:0/2221732708 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1065+0+0 (secure 0 0 0) 0x7fb2700423e0 con 0x7fb2880f4c00 2024-02-20T13:01:29.229 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:29.225+0000 7fb285d9b700 1 --2- 172.21.15.71:0/2221732708 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fb274031090 0x7fb274033550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T13:01:29.229 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:29.225+0000 7fb285d9b700 1 --2- 172.21.15.71:0/2221732708 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fb274031090 0x7fb274033550 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 rx=0x7fb2780096a0 tx=0x7fb278006b40).ready entity=mgr.14156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T13:01:29.231 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:29.229+0000 7fb27effd700 1 -- 172.21.15.71:0/2221732708 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+153237 (secure 0 0 0) 0x7fb27001d7f0 con 0x7fb2880f4c00 2024-02-20T13:01:29.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 13:01:29 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T13:01:27.808250+0000 mgr.smithi064.vriyor (mgr.14156) 328 : cluster [DBG] pgmap v311: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T13:01:29.437 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:29.436+0000 7fb28ceda700 1 -- 172.21.15.71:0/2221732708 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fb288045bb0 con 0x7fb2880f4c00 2024-02-20T13:01:29.438 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:29.436+0000 7fb27effd700 1 -- 172.21.15.71:0/2221732708 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7fb27001d7f0 con 0x7fb2880f4c00 2024-02-20T13:01:29.438 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-20T13:01:29.438 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":1,"fsid":"680ee126-cfee-11ee-95bb-87774f69a715","modified":"2024-02-20T12:49:44.699059Z","created":"2024-02-20T12:49:44.699059Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi064","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:3300","nonce":0},{"type":"v1","addr":"172.21.15.64:6789","nonce":0}]},"addr":"172.21.15.64:6789/0","public_addr":"172.21.15.64:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-20T13:01:29.440 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:29.438+0000 7fb28ceda700 1 -- 172.21.15.71:0/2221732708 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fb274031090 msgr2=0x7fb274033550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T13:01:29.440 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:29.438+0000 7fb28ceda700 1 --2- 172.21.15.71:0/2221732708 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fb274031090 0x7fb274033550 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 rx=0x7fb2780096a0 tx=0x7fb278006b40).stop 2024-02-20T13:01:29.440 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:29.438+0000 7fb28ceda700 1 -- 172.21.15.71:0/2221732708 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb2880f4c00 msgr2=0x7fb288109500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T13:01:29.440 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:29.438+0000 7fb28ceda700 1 --2- 172.21.15.71:0/2221732708 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb2880f4c00 0x7fb288109500 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 rx=0x7fb2700089f0 tx=0x7fb27000b040).stop 2024-02-20T13:01:29.440 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:29.438+0000 7fb28ceda700 1 -- 172.21.15.71:0/2221732708 shutdown_connections 2024-02-20T13:01:29.441 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:29.438+0000 7fb28ceda700 1 --2- 172.21.15.71:0/2221732708 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fb274031090 0x7fb274033550 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T13:01:29.441 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:29.438+0000 7fb28ceda700 1 --2- 172.21.15.71:0/2221732708 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb2880f4c00 0x7fb288109500 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T13:01:29.441 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:29.438+0000 7fb28ceda700 1 -- 172.21.15.71:0/2221732708 >> 172.21.15.71:0/2221732708 conn(0x7fb2880f0360 msgr2=0x7fb2880f2690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T13:01:29.441 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:29.438+0000 7fb28ceda700 1 -- 172.21.15.71:0/2221732708 shutdown_connections 2024-02-20T13:01:29.441 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:29.438+0000 7fb28ceda700 1 -- 172.21.15.71:0/2221732708 wait complete. 2024-02-20T13:01:29.441 INFO:teuthology.orchestra.run.smithi071.stderr:dumped monmap epoch 1 2024-02-20T13:01:30.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 13:01:30 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T13:01:29.438324+0000 mon.smithi064 (mon.0) 295 : audit [DBG] from='client.? 172.21.15.71:0/2221732708' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-20T13:01:31.162 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-20T13:01:31.163 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/daemon-base:latest-pacific shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 680ee126-cfee-11ee-95bb-87774f69a715 -- ceph mon dump -f json 2024-02-20T13:01:31.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 13:01:31 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T13:01:29.808628+0000 mgr.smithi064.vriyor (mgr.14156) 329 : cluster [DBG] pgmap v312: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T13:01:33.081 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:33.079+0000 7efea0642700 1 -- 172.21.15.71:0/2146634687 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efe980f5480 msgr2=0x7efe980f58a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T13:01:33.082 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:33.079+0000 7efea0642700 1 --2- 172.21.15.71:0/2146634687 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efe980f5480 0x7efe980f58a0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 rx=0x7efe94009fb0 tx=0x7efe94009b30).stop 2024-02-20T13:01:33.082 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:33.079+0000 7efea0642700 1 -- 172.21.15.71:0/2146634687 shutdown_connections 2024-02-20T13:01:33.082 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:33.079+0000 7efea0642700 1 --2- 172.21.15.71:0/2146634687 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efe980f5480 0x7efe980f58a0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T13:01:33.082 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:33.079+0000 7efea0642700 1 -- 172.21.15.71:0/2146634687 >> 172.21.15.71:0/2146634687 conn(0x7efe980f0be0 msgr2=0x7efe980f3000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T13:01:33.082 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:33.079+0000 7efea0642700 1 -- 172.21.15.71:0/2146634687 shutdown_connections 2024-02-20T13:01:33.082 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:33.079+0000 7efea0642700 1 -- 172.21.15.71:0/2146634687 wait complete. 2024-02-20T13:01:33.082 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:33.080+0000 7efea0642700 1 Processor -- start 2024-02-20T13:01:33.082 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:33.080+0000 7efea0642700 1 -- start start 2024-02-20T13:01:33.082 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:33.080+0000 7efea0642700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efe980f5480 0x7efe980fd460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T13:01:33.082 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:33.080+0000 7efea0642700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe980fb8e0 con 0x7efe980f5480 2024-02-20T13:01:33.083 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:33.080+0000 7efe9e3de700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efe980f5480 0x7efe980fd460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T13:01:33.083 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:33.080+0000 7efe9e3de700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efe980f5480 0x7efe980fd460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.71:38086/0 (socket says 172.21.15.71:38086) 2024-02-20T13:01:33.083 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:33.080+0000 7efe9e3de700 1 -- 172.21.15.71:0/2086472516 learned_addr learned my addr 172.21.15.71:0/2086472516 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-02-20T13:01:33.083 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:33.081+0000 7efe9e3de700 1 -- 172.21.15.71:0/2086472516 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe94005040 con 0x7efe980f5480 2024-02-20T13:01:33.083 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:33.081+0000 7efe9e3de700 1 --2- 172.21.15.71:0/2086472516 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efe980f5480 0x7efe980fd460 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 rx=0x7efe94000c00 tx=0x7efe9400b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T13:01:33.083 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:33.081+0000 7efe8effd700 1 -- 172.21.15.71:0/2086472516 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7efe940095f0 con 0x7efe980f5480 2024-02-20T13:01:33.083 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:33.081+0000 7efea0642700 1 -- 172.21.15.71:0/2086472516 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe980fba80 con 0x7efe980f5480 2024-02-20T13:01:33.083 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:33.081+0000 7efe8effd700 1 -- 172.21.15.71:0/2086472516 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(25 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7efe94007e50 con 0x7efe980f5480 2024-02-20T13:01:33.083 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:33.081+0000 7efe8effd700 1 -- 172.21.15.71:0/2086472516 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7efe94014e70 con 0x7efe980f5480 2024-02-20T13:01:33.083 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:33.081+0000 7efea0642700 1 -- 172.21.15.71:0/2086472516 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe980fbf10 con 0x7efe980f5480 2024-02-20T13:01:33.084 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:33.082+0000 7efe8effd700 1 -- 172.21.15.71:0/2086472516 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 37151+0+0 (secure 0 0 0) 0x7efe94025070 con 0x7efe980f5480 2024-02-20T13:01:33.085 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:33.082+0000 7efe8effd700 1 --2- 172.21.15.71:0/2086472516 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7efe84031090 0x7efe84033550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T13:01:33.085 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:33.082+0000 7efea0642700 1 -- 172.21.15.71:0/2086472516 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe980591e0 con 0x7efe980f5480 2024-02-20T13:01:33.085 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:33.082+0000 7efe8effd700 1 -- 172.21.15.71:0/2086472516 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1065+0+0 (secure 0 0 0) 0x7efe94041dc0 con 0x7efe980f5480 2024-02-20T13:01:33.085 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:33.082+0000 7efe9dbdd700 1 --2- 172.21.15.71:0/2086472516 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7efe84031090 0x7efe84033550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T13:01:33.085 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:33.083+0000 7efe9dbdd700 1 --2- 172.21.15.71:0/2086472516 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7efe84031090 0x7efe84033550 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 rx=0x7efe880096a0 tx=0x7efe88006b40).ready entity=mgr.14156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T13:01:33.087 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:33.087+0000 7efe8effd700 1 -- 172.21.15.71:0/2086472516 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+153237 (secure 0 0 0) 0x7efe940079c0 con 0x7efe980f5480 2024-02-20T13:01:33.296 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:33.294+0000 7efea0642700 1 -- 172.21.15.71:0/2086472516 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7efe98045bb0 con 0x7efe980f5480 2024-02-20T13:01:33.297 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:33.294+0000 7efe8effd700 1 -- 172.21.15.71:0/2086472516 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7efe9400f300 con 0x7efe980f5480 2024-02-20T13:01:33.297 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-20T13:01:33.297 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":1,"fsid":"680ee126-cfee-11ee-95bb-87774f69a715","modified":"2024-02-20T12:49:44.699059Z","created":"2024-02-20T12:49:44.699059Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi064","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:3300","nonce":0},{"type":"v1","addr":"172.21.15.64:6789","nonce":0}]},"addr":"172.21.15.64:6789/0","public_addr":"172.21.15.64:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-20T13:01:33.298 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:33.296+0000 7efea0642700 1 -- 172.21.15.71:0/2086472516 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7efe84031090 msgr2=0x7efe84033550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T13:01:33.298 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:33.296+0000 7efea0642700 1 --2- 172.21.15.71:0/2086472516 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7efe84031090 0x7efe84033550 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 rx=0x7efe880096a0 tx=0x7efe88006b40).stop 2024-02-20T13:01:33.298 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:33.296+0000 7efea0642700 1 -- 172.21.15.71:0/2086472516 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efe980f5480 msgr2=0x7efe980fd460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T13:01:33.298 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:33.296+0000 7efea0642700 1 --2- 172.21.15.71:0/2086472516 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efe980f5480 0x7efe980fd460 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 rx=0x7efe94000c00 tx=0x7efe9400b040).stop 2024-02-20T13:01:33.298 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:33.296+0000 7efea0642700 1 -- 172.21.15.71:0/2086472516 shutdown_connections 2024-02-20T13:01:33.298 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:33.296+0000 7efea0642700 1 --2- 172.21.15.71:0/2086472516 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7efe84031090 0x7efe84033550 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T13:01:33.298 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:33.296+0000 7efea0642700 1 --2- 172.21.15.71:0/2086472516 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efe980f5480 0x7efe980fd460 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T13:01:33.298 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:33.296+0000 7efea0642700 1 -- 172.21.15.71:0/2086472516 >> 172.21.15.71:0/2086472516 conn(0x7efe980f0be0 msgr2=0x7efe9805ff40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T13:01:33.298 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:33.296+0000 7efea0642700 1 -- 172.21.15.71:0/2086472516 shutdown_connections 2024-02-20T13:01:33.298 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:33.296+0000 7efea0642700 1 -- 172.21.15.71:0/2086472516 wait complete. 2024-02-20T13:01:33.298 INFO:teuthology.orchestra.run.smithi071.stderr:dumped monmap epoch 1 2024-02-20T13:01:33.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 13:01:33 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T13:01:31.808954+0000 mgr.smithi064.vriyor (mgr.14156) 330 : cluster [DBG] pgmap v313: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T13:01:34.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 13:01:34 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T13:01:33.295242+0000 mon.smithi064 (mon.0) 296 : audit [DBG] from='client.? 172.21.15.71:0/2086472516' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-20T13:01:34.908 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-20T13:01:34.909 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/daemon-base:latest-pacific shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 680ee126-cfee-11ee-95bb-87774f69a715 -- ceph mon dump -f json 2024-02-20T13:01:35.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 13:01:35 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T13:01:33.809238+0000 mgr.smithi064.vriyor (mgr.14156) 331 : cluster [DBG] pgmap v314: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T13:01:36.977 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:36.976+0000 7fc62d7cc700 1 -- 172.21.15.71:0/2351150253 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc628060360 msgr2=0x7fc6280f8fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T13:01:36.977 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:36.976+0000 7fc62d7cc700 1 --2- 172.21.15.71:0/2351150253 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc628060360 0x7fc6280f8fb0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 rx=0x7fc618009fb0 tx=0x7fc618009b30).stop 2024-02-20T13:01:36.977 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:36.976+0000 7fc62d7cc700 1 -- 172.21.15.71:0/2351150253 shutdown_connections 2024-02-20T13:01:36.977 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:36.976+0000 7fc62d7cc700 1 --2- 172.21.15.71:0/2351150253 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc628060360 0x7fc6280f8fb0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T13:01:36.977 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:36.976+0000 7fc62d7cc700 1 -- 172.21.15.71:0/2351150253 >> 172.21.15.71:0/2351150253 conn(0x7fc6280f0a30 msgr2=0x7fc6280f2e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T13:01:36.977 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:36.976+0000 7fc62d7cc700 1 -- 172.21.15.71:0/2351150253 shutdown_connections 2024-02-20T13:01:36.977 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:36.976+0000 7fc62d7cc700 1 -- 172.21.15.71:0/2351150253 wait complete. 2024-02-20T13:01:36.977 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:36.977+0000 7fc62d7cc700 1 Processor -- start 2024-02-20T13:01:36.979 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:36.977+0000 7fc62d7cc700 1 -- start start 2024-02-20T13:01:36.979 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:36.977+0000 7fc62d7cc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc628060360 0x7fc6281065b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T13:01:36.979 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:36.977+0000 7fc62d7cc700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc628106aa0 con 0x7fc628060360 2024-02-20T13:01:36.979 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:36.977+0000 7fc627fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc628060360 0x7fc6281065b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T13:01:36.979 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:36.977+0000 7fc627fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc628060360 0x7fc6281065b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.71:38100/0 (socket says 172.21.15.71:38100) 2024-02-20T13:01:36.980 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:36.977+0000 7fc627fff700 1 -- 172.21.15.71:0/2486274179 learned_addr learned my addr 172.21.15.71:0/2486274179 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-02-20T13:01:36.980 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:36.977+0000 7fc627fff700 1 -- 172.21.15.71:0/2486274179 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc618005040 con 0x7fc628060360 2024-02-20T13:01:36.980 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:36.978+0000 7fc627fff700 1 --2- 172.21.15.71:0/2486274179 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc628060360 0x7fc6281065b0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 rx=0x7fc618009f80 tx=0x7fc61800b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T13:01:36.980 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:36.978+0000 7fc624ff9700 1 -- 172.21.15.71:0/2486274179 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc618009320 con 0x7fc628060360 2024-02-20T13:01:36.980 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:36.978+0000 7fc624ff9700 1 -- 172.21.15.71:0/2486274179 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(25 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7fc618013680 con 0x7fc628060360 2024-02-20T13:01:36.980 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:36.978+0000 7fc62d7cc700 1 -- 172.21.15.71:0/2486274179 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc628106ca0 con 0x7fc628060360 2024-02-20T13:01:36.980 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:36.978+0000 7fc624ff9700 1 -- 172.21.15.71:0/2486274179 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc61801eda0 con 0x7fc628060360 2024-02-20T13:01:36.980 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:36.978+0000 7fc62d7cc700 1 -- 172.21.15.71:0/2486274179 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc628107130 con 0x7fc628060360 2024-02-20T13:01:36.980 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:36.978+0000 7fc624ff9700 1 -- 172.21.15.71:0/2486274179 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 37151+0+0 (secure 0 0 0) 0x7fc618014420 con 0x7fc628060360 2024-02-20T13:01:36.980 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:36.979+0000 7fc624ff9700 1 --2- 172.21.15.71:0/2486274179 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fc610031090 0x7fc610033550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T13:01:36.980 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:36.979+0000 7fc624ff9700 1 -- 172.21.15.71:0/2486274179 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1065+0+0 (secure 0 0 0) 0x7fc618042420 con 0x7fc628060360 2024-02-20T13:01:36.980 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:36.979+0000 7fc6277fe700 1 --2- 172.21.15.71:0/2486274179 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fc610031090 0x7fc610033550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T13:01:36.980 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:36.979+0000 7fc62d7cc700 1 -- 172.21.15.71:0/2486274179 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6280591e0 con 0x7fc628060360 2024-02-20T13:01:36.981 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:36.979+0000 7fc6277fe700 1 --2- 172.21.15.71:0/2486274179 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fc610031090 0x7fc610033550 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 rx=0x7fc61c0096a0 tx=0x7fc61c006b40).ready entity=mgr.14156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T13:01:36.984 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:36.983+0000 7fc624ff9700 1 -- 172.21.15.71:0/2486274179 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+153237 (secure 0 0 0) 0x7fc618014700 con 0x7fc628060360 2024-02-20T13:01:37.204 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:37.203+0000 7fc62d7cc700 1 -- 172.21.15.71:0/2486274179 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fc628045bb0 con 0x7fc628060360 2024-02-20T13:01:37.204 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:37.203+0000 7fc624ff9700 1 -- 172.21.15.71:0/2486274179 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7fc61801c320 con 0x7fc628060360 2024-02-20T13:01:37.204 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-20T13:01:37.204 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":1,"fsid":"680ee126-cfee-11ee-95bb-87774f69a715","modified":"2024-02-20T12:49:44.699059Z","created":"2024-02-20T12:49:44.699059Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi064","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:3300","nonce":0},{"type":"v1","addr":"172.21.15.64:6789","nonce":0}]},"addr":"172.21.15.64:6789/0","public_addr":"172.21.15.64:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-20T13:01:37.206 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:37.205+0000 7fc62d7cc700 1 -- 172.21.15.71:0/2486274179 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fc610031090 msgr2=0x7fc610033550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T13:01:37.206 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:37.205+0000 7fc62d7cc700 1 --2- 172.21.15.71:0/2486274179 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fc610031090 0x7fc610033550 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 rx=0x7fc61c0096a0 tx=0x7fc61c006b40).stop 2024-02-20T13:01:37.206 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:37.205+0000 7fc62d7cc700 1 -- 172.21.15.71:0/2486274179 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc628060360 msgr2=0x7fc6281065b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T13:01:37.206 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:37.205+0000 7fc62d7cc700 1 --2- 172.21.15.71:0/2486274179 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc628060360 0x7fc6281065b0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 rx=0x7fc618009f80 tx=0x7fc61800b040).stop 2024-02-20T13:01:37.206 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:37.206+0000 7fc62d7cc700 1 -- 172.21.15.71:0/2486274179 shutdown_connections 2024-02-20T13:01:37.206 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:37.206+0000 7fc62d7cc700 1 --2- 172.21.15.71:0/2486274179 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fc610031090 0x7fc610033550 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T13:01:37.206 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:37.206+0000 7fc62d7cc700 1 --2- 172.21.15.71:0/2486274179 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc628060360 0x7fc6281065b0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T13:01:37.207 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:37.206+0000 7fc62d7cc700 1 -- 172.21.15.71:0/2486274179 >> 172.21.15.71:0/2486274179 conn(0x7fc6280f0a30 msgr2=0x7fc6280f14e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T13:01:37.207 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:37.206+0000 7fc62d7cc700 1 -- 172.21.15.71:0/2486274179 shutdown_connections 2024-02-20T13:01:37.207 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:37.206+0000 7fc62d7cc700 1 -- 172.21.15.71:0/2486274179 wait complete. 2024-02-20T13:01:37.207 INFO:teuthology.orchestra.run.smithi071.stderr:dumped monmap epoch 1 2024-02-20T13:01:37.313 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 13:01:37 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T13:01:35.809612+0000 mgr.smithi064.vriyor (mgr.14156) 332 : cluster [DBG] pgmap v315: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T13:01:37.313 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 13:01:37 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cephadm 2024-02-20T13:01:36.149954+0000 mgr.smithi064.vriyor (mgr.14156) 333 : cephadm [ERR] cephadm exited with an error code: 1, stderr:Non-zero exit code 125 from /bin/podman container inspect --format {{.State.Status}} ceph-680ee126-cfee-11ee-95bb-87774f69a715-prometheus.smithi064 2024-02-20T13:01:37.313 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 13:01:37 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: /bin/podman: stderr Error: no such container ceph-680ee126-cfee-11ee-95bb-87774f69a715-prometheus.smithi064 2024-02-20T13:01:37.313 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 13:01:37 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: Deploy daemon prometheus.smithi064 ... 2024-02-20T13:01:37.313 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 13:01:37 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: Non-zero exit code 125 from /bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint stat --init -e CONTAINER_IMAGE=docker.io/prom/prometheus:v2.18.1 -e NODE_NAME=smithi064 -e CEPH_USE_RANDOM_NONCE=1 docker.io/prom/prometheus:v2.18.1 -c %u %g /etc/prometheus 2024-02-20T13:01:37.313 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 13:01:37 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: stat: stderr Trying to pull docker.io/prom/prometheus:v2.18.1... 2024-02-20T13:01:37.313 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 13:01:37 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: stat: stderr time="2024-02-20T12:59:43Z" level=warning msg="Failed, retrying in 1s ... (1/3). Error: copying system image from manifest list: determining manifest MIME type for docker://prom/prometheus:v2.18.1: reading manifest sha256:b4e6cd0275a26750505e539f8528e891053434ebd3972be02645bed5f02f0795 in docker.io/prom/prometheus: toomanyrequests: You have reached your pull rate limit. You may increase the limit by authenticating and upgrading: https://www.docker.com/increase-rate-limit" 2024-02-20T13:01:37.314 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 13:01:37 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: stat: stderr time="2024-02-20T13:00:20Z" level=warning msg="Failed, retrying in 1s ... (2/3). Error: initializing source docker://prom/prometheus:v2.18.1: (Mirrors also failed: [docker-mirror.front.sepia.ceph.com:5000/prom/prometheus:v2.18.1: pinging container registry docker-mirror.front.sepia.ceph.com:5000: Get \"https://docker-mirror.front.sepia.ceph.com:5000/v2/\": dial tcp 172.21.0.79:5000: connect: no route to host]): docker.io/prom/prometheus:v2.18.1: reading manifest v2.18.1 in docker.io/prom/prometheus: toomanyrequests: You have reached your pull rate limit. You may increase the limit by authenticating and upgrading: https://www.docker.com/increase-rate-limit" 2024-02-20T13:01:37.314 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 13:01:37 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: stat: stderr time="2024-02-20T13:00:58Z" level=warning msg="Failed, retrying in 1s ... (3/3). Error: initializing source docker://prom/prometheus:v2.18.1: (Mirrors also failed: [docker-mirror.front.sepia.ceph.com:5000/prom/prometheus:v2.18.1: pinging container registry docker-mirror.front.sepia.ceph.com:5000: Get \"https://docker-mirror.front.sepia.ceph.com:5000/v2/\": dial tcp 172.21.0.79:5000: connect: no route to host]): docker.io/prom/prometheus:v2.18.1: reading manifest v2.18.1 in docker.io/prom/prometheus: toomanyrequests: You have reached your pull rate limit. You may increase the limit by authenticating and upgrading: https://www.docker.com/increase-rate-limit" 2024-02-20T13:01:37.314 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 13:01:37 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: stat: stderr Error: copying system image from manifest list: determining manifest MIME type for docker://prom/prometheus:v2.18.1: reading manifest sha256:b4e6cd0275a26750505e539f8528e891053434ebd3972be02645bed5f02f0795 in docker.io/prom/prometheus: toomanyrequests: You have reached your pull rate limit. You may increase the limit by authenticating and upgrading: https://www.docker.com/increase-rate-limit 2024-02-20T13:01:37.314 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 13:01:37 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: Traceback (most recent call last): 2024-02-20T13:01:37.314 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 13:01:37 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: File "/var/lib/ceph/680ee126-cfee-11ee-95bb-87774f69a715/cephadm.d4237e4639c108308fe13147b1c08af93c3d5724d9ff21ae797eb4b78fea3931", line 8230, in 2024-02-20T13:01:37.314 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 13:01:37 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: main() 2024-02-20T13:01:37.314 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 13:01:37 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: File "/var/lib/ceph/680ee126-cfee-11ee-95bb-87774f69a715/cephadm.d4237e4639c108308fe13147b1c08af93c3d5724d9ff21ae797eb4b78fea3931", line 8218, in main 2024-02-20T13:01:37.314 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 13:01:37 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: r = ctx.func(ctx) 2024-02-20T13:01:37.314 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 13:01:37 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: File "/var/lib/ceph/680ee126-cfee-11ee-95bb-87774f69a715/cephadm.d4237e4639c108308fe13147b1c08af93c3d5724d9ff21ae797eb4b78fea3931", line 1759, in _default_image 2024-02-20T13:01:37.314 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 13:01:37 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: return func(ctx) 2024-02-20T13:01:37.314 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 13:01:37 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: File "/var/lib/ceph/680ee126-cfee-11ee-95bb-87774f69a715/cephadm.d4237e4639c108308fe13147b1c08af93c3d5724d9ff21ae797eb4b78fea3931", line 4344, in command_deploy 2024-02-20T13:01:37.314 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 13:01:37 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: uid, gid = extract_uid_gid_monitoring(ctx, daemon_type) 2024-02-20T13:01:37.314 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 13:01:37 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: File "/var/lib/ceph/680ee126-cfee-11ee-95bb-87774f69a715/cephadm.d4237e4639c108308fe13147b1c08af93c3d5724d9ff21ae797eb4b78fea3931", line 4269, in extract_uid_gid_monitoring 2024-02-20T13:01:37.314 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 13:01:37 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: uid, gid = extract_uid_gid(ctx, file_path='/etc/prometheus') 2024-02-20T13:01:37.315 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 13:01:37 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: File "/var/lib/ceph/680ee126-cfee-11ee-95bb-87774f69a715/cephadm.d4237e4639c108308fe13147b1c08af93c3d5724d9ff21ae797eb4b78fea3931", line 2548, in extract_uid_gid 2024-02-20T13:01:37.315 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 13:01:37 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: raise RuntimeError('uid/gid not found') 2024-02-20T13:01:37.315 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 13:01:37 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: RuntimeError: uid/gid not found 2024-02-20T13:01:37.315 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 13:01:37 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: Traceback (most recent call last): 2024-02-20T13:01:37.315 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 13:01:37 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: File "/usr/share/ceph/mgr/cephadm/serve.py", line 1347, in _remote_connection 2024-02-20T13:01:37.315 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 13:01:37 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: yield (conn, connr) 2024-02-20T13:01:37.315 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 13:01:37 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: File "/usr/share/ceph/mgr/cephadm/serve.py", line 1244, in _run_cephadm 2024-02-20T13:01:37.315 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 13:01:37 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: code, '\n'.join(err))) 2024-02-20T13:01:37.315 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 13:01:37 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: orchestrator._interface.OrchestratorError: cephadm exited with an error code: 1, stderr:Non-zero exit code 125 from /bin/podman container inspect --format {{.State.Status}} ceph-680ee126-cfee-11ee-95bb-87774f69a715-prometheus.smithi064 2024-02-20T13:01:37.315 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 13:01:37 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: /bin/podman: stderr Error: no such container ceph-680ee126-cfee-11ee-95bb-87774f69a715-prometheus.smithi064 2024-02-20T13:01:37.315 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 13:01:37 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: Deploy daemon prometheus.smithi064 ... 2024-02-20T13:01:37.315 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 13:01:37 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: Non-zero exit code 125 from /bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint stat --init -e CONTAINER_IMAGE=docker.io/prom/prometheus:v2.18.1 -e NODE_NAME=smithi064 -e CEPH_USE_RANDOM_NONCE=1 docker.io/prom/prometheus:v2.18.1 -c %u %g /etc/prometheus 2024-02-20T13:01:37.315 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 13:01:37 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: stat: stderr Trying to pull docker.io/prom/prometheus:v2.18.1... 2024-02-20T13:01:37.315 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 13:01:37 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: stat: stderr time="2024-02-20T12:59:43Z" level=warning msg="Failed, retrying in 1s ... (1/3). Error: copying system image from manifest list: determining manifest MIME type for docker://prom/prometheus:v2.18.1: reading manifest sha256:b4e6cd0275a26750505e539f8528e891053434ebd3972be02645bed5f02f0795 in docker.io/prom/prometheus: toomanyrequests: You have reached your pull rate limit. You may increase the limit by authenticating and upgrading: https://www.docker.com/increase-rate-limit" 2024-02-20T13:01:37.315 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 13:01:37 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: stat: stderr time="2024-02-20T13:00:20Z" level=warning msg="Failed, retrying in 1s ... (2/3). Error: initializing source docker://prom/prometheus:v2.18.1: (Mirrors also failed: [docker-mirror.front.sepia.ceph.com:5000/prom/prometheus:v2.18.1: pinging container registry docker-mirror.front.sepia.ceph.com:5000: Get \"https://docker-mirror.front.sepia.ceph.com:5000/v2/\": dial tcp 172.21.0.79:5000: connect: no route to host]): docker.io/prom/prometheus:v2.18.1: reading manifest v2.18.1 in docker.io/prom/prometheus: toomanyrequests: You have reached your pull rate limit. You may increase the limit by authenticating and upgrading: https://www.docker.com/increase-rate-limit" 2024-02-20T13:01:37.316 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 13:01:37 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: stat: stderr time="2024-02-20T13:00:58Z" level=warning msg="Failed, retrying in 1s ... (3/3). Error: initializing source docker://prom/prometheus:v2.18.1: (Mirrors also failed: [docker-mirror.front.sepia.ceph.com:5000/prom/prometheus:v2.18.1: pinging container registry docker-mirror.front.sepia.ceph.com:5000: Get \"https://docker-mirror.front.sepia.ceph.com:5000/v2/\": dial tcp 172.21.0.79:5000: connect: no route to host]): docker.io/prom/prometheus:v2.18.1: reading manifest v2.18.1 in docker.io/prom/prometheus: toomanyrequests: You have reached your pull rate limit. You may increase the limit by authenticating and upgrading: https://www.docker.com/increase-rate-limit" 2024-02-20T13:01:37.316 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 13:01:37 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: stat: stderr Error: copying system image from manifest list: determining manifest MIME type for docker://prom/prometheus:v2.18.1: reading manifest sha256:b4e6cd0275a26750505e539f8528e891053434ebd3972be02645bed5f02f0795 in docker.io/prom/prometheus: toomanyrequests: You have reached your pull rate limit. You may increase the limit by authenticating and upgrading: https://www.docker.com/increase-rate-limit 2024-02-20T13:01:37.316 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 13:01:37 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: Traceback (most recent call last): 2024-02-20T13:01:37.316 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 13:01:37 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: File "/var/lib/ceph/680ee126-cfee-11ee-95bb-87774f69a715/cephadm.d4237e4639c108308fe13147b1c08af93c3d5724d9ff21ae797eb4b78fea3931", line 8230, in 2024-02-20T13:01:37.316 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 13:01:37 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: main() 2024-02-20T13:01:37.316 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 13:01:37 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: File "/var/lib/ceph/680ee126-cfee-11ee-95bb-87774f69a715/cephadm.d4237e4639c108308fe13147b1c08af93c3d5724d9ff21ae797eb4b78fea3931", line 8218, in main 2024-02-20T13:01:37.316 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 13:01:37 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: r = ctx.func(ctx) 2024-02-20T13:01:37.316 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 13:01:37 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: File "/var/lib/ceph/680ee126-cfee-11ee-95bb-87774f69a715/cephadm.d4237e4639c108308fe13147b1c08af93c3d5724d9ff21ae797eb4b78fea3931", line 1759, in _default_image 2024-02-20T13:01:37.316 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 13:01:37 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: return func(ctx) 2024-02-20T13:01:37.316 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 13:01:37 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: File "/var/lib/ceph/680ee126-cfee-11ee-95bb-87774f69a715/cephadm.d4237e4639c108308fe13147b1c08af93c3d5724d9ff21ae797eb4b78fea3931", line 4344, in command_deploy 2024-02-20T13:01:37.316 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 13:01:37 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: uid, gid = extract_uid_gid_monitoring(ctx, daemon_type) 2024-02-20T13:01:37.316 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 13:01:37 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: File "/var/lib/ceph/680ee126-cfee-11ee-95bb-87774f69a715/cephadm.d4237e4639c108308fe13147b1c08af93c3d5724d9ff21ae797eb4b78fea3931", line 4269, in extract_uid_gid_monitoring 2024-02-20T13:01:37.316 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 13:01:37 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: uid, gid = extract_uid_gid(ctx, file_path='/etc/prometheus') 2024-02-20T13:01:37.316 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 13:01:37 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: File "/var/lib/ceph/680ee126-cfee-11ee-95bb-87774f69a715/cephadm.d4237e4639c108308fe13147b1c08af93c3d5724d9ff21ae797eb4b78fea3931", line 2548, in extract_uid_gid 2024-02-20T13:01:37.317 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 13:01:37 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: raise RuntimeError('uid/gid not found') 2024-02-20T13:01:37.317 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 13:01:37 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: RuntimeError: uid/gid not found 2024-02-20T13:01:37.317 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 13:01:37 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cephadm 2024-02-20T13:01:36.150223+0000 mgr.smithi064.vriyor (mgr.14156) 334 : cephadm [ERR] Failed while placing prometheus.smithi064 on smithi064: cephadm exited with an error code: 1, stderr:Non-zero exit code 125 from /bin/podman container inspect --format {{.State.Status}} ceph-680ee126-cfee-11ee-95bb-87774f69a715-prometheus.smithi064 2024-02-20T13:01:37.317 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 13:01:37 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: /bin/podman: stderr Error: no such container ceph-680ee126-cfee-11ee-95bb-87774f69a715-prometheus.smithi064 2024-02-20T13:01:37.317 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 13:01:37 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: Deploy daemon prometheus.smithi064 ... 2024-02-20T13:01:37.317 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 13:01:37 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: Non-zero exit code 125 from /bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint stat --init -e CONTAINER_IMAGE=docker.io/prom/prometheus:v2.18.1 -e NODE_NAME=smithi064 -e CEPH_USE_RANDOM_NONCE=1 docker.io/prom/prometheus:v2.18.1 -c %u %g /etc/prometheus 2024-02-20T13:01:37.317 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 13:01:37 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: stat: stderr Trying to pull docker.io/prom/prometheus:v2.18.1... 2024-02-20T13:01:37.317 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 13:01:37 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: stat: stderr time="2024-02-20T12:59:43Z" level=warning msg="Failed, retrying in 1s ... (1/3). Error: copying system image from manifest list: determining manifest MIME type for docker://prom/prometheus:v2.18.1: reading manifest sha256:b4e6cd0275a26750505e539f8528e891053434ebd3972be02645bed5f02f0795 in docker.io/prom/prometheus: toomanyrequests: You have reached your pull rate limit. You may increase the limit by authenticating and upgrading: https://www.docker.com/increase-rate-limit" 2024-02-20T13:01:37.317 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 13:01:37 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: stat: stderr time="2024-02-20T13:00:20Z" level=warning msg="Failed, retrying in 1s ... (2/3). Error: initializing source docker://prom/prometheus:v2.18.1: (Mirrors also failed: [docker-mirror.front.sepia.ceph.com:5000/prom/prometheus:v2.18.1: pinging container registry docker-mirror.front.sepia.ceph.com:5000: Get \"https://docker-mirror.front.sepia.ceph.com:5000/v2/\": dial tcp 172.21.0.79:5000: connect: no route to host]): docker.io/prom/prometheus:v2.18.1: reading manifest v2.18.1 in docker.io/prom/prometheus: toomanyrequests: You have reached your pull rate limit. You may increase the limit by authenticating and upgrading: https://www.docker.com/increase-rate-limit" 2024-02-20T13:01:37.317 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 13:01:37 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: stat: stderr time="2024-02-20T13:00:58Z" level=warning msg="Failed, retrying in 1s ... (3/3). Error: initializing source docker://prom/prometheus:v2.18.1: (Mirrors also failed: [docker-mirror.front.sepia.ceph.com:5000/prom/prometheus:v2.18.1: pinging container registry docker-mirror.front.sepia.ceph.com:5000: Get \"https://docker-mirror.front.sepia.ceph.com:5000/v2/\": dial tcp 172.21.0.79:5000: connect: no route to host]): docker.io/prom/prometheus:v2.18.1: reading manifest v2.18.1 in docker.io/prom/prometheus: toomanyrequests: You have reached your pull rate limit. You may increase the limit by authenticating and upgrading: https://www.docker.com/increase-rate-limit" 2024-02-20T13:01:37.317 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 13:01:37 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: stat: stderr Error: copying system image from manifest list: determining manifest MIME type for docker://prom/prometheus:v2.18.1: reading manifest sha256:b4e6cd0275a26750505e539f8528e891053434ebd3972be02645bed5f02f0795 in docker.io/prom/prometheus: toomanyrequests: You have reached your pull rate limit. You may increase the limit by authenticating and upgrading: https://www.docker.com/increase-rate-limit 2024-02-20T13:01:37.317 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 13:01:37 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: Traceback (most recent call last): 2024-02-20T13:01:37.317 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 13:01:37 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: File "/var/lib/ceph/680ee126-cfee-11ee-95bb-87774f69a715/cephadm.d4237e4639c108308fe13147b1c08af93c3d5724d9ff21ae797eb4b78fea3931", line 8230, in 2024-02-20T13:01:37.317 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 13:01:37 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: main() 2024-02-20T13:01:37.317 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 13:01:37 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: File "/var/lib/ceph/680ee126-cfee-11ee-95bb-87774f69a715/cephadm.d4237e4639c108308fe13147b1c08af93c3d5724d9ff21ae797eb4b78fea3931", line 8218, in main 2024-02-20T13:01:37.318 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 13:01:37 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: r = ctx.func(ctx) 2024-02-20T13:01:37.318 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 13:01:37 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: File "/var/lib/ceph/680ee126-cfee-11ee-95bb-87774f69a715/cephadm.d4237e4639c108308fe13147b1c08af93c3d5724d9ff21ae797eb4b78fea3931", line 1759, in _default_image 2024-02-20T13:01:37.318 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 13:01:37 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: return func(ctx) 2024-02-20T13:01:37.318 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 13:01:37 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: File "/var/lib/ceph/680ee126-cfee-11ee-95bb-87774f69a715/cephadm.d4237e4639c108308fe13147b1c08af93c3d5724d9ff21ae797eb4b78fea3931", line 4344, in command_deploy 2024-02-20T13:01:37.318 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 13:01:37 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: uid, gid = extract_uid_gid_monitoring(ctx, daemon_type) 2024-02-20T13:01:37.318 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 13:01:37 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: File "/var/lib/ceph/680ee126-cfee-11ee-95bb-87774f69a715/cephadm.d4237e4639c108308fe13147b1c08af93c3d5724d9ff21ae797eb4b78fea3931", line 4269, in extract_uid_gid_monitoring 2024-02-20T13:01:37.318 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 13:01:37 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: uid, gid = extract_uid_gid(ctx, file_path='/etc/prometheus') 2024-02-20T13:01:37.318 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 13:01:37 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: File "/var/lib/ceph/680ee126-cfee-11ee-95bb-87774f69a715/cephadm.d4237e4639c108308fe13147b1c08af93c3d5724d9ff21ae797eb4b78fea3931", line 2548, in extract_uid_gid 2024-02-20T13:01:37.318 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 13:01:37 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: raise RuntimeError('uid/gid not found') 2024-02-20T13:01:37.318 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 13:01:37 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: RuntimeError: uid/gid not found 2024-02-20T13:01:37.318 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 13:01:37 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T13:01:36.151320+0000 mon.smithi064 (mon.0) 297 : audit [DBG] from='mgr.14156 172.21.15.64:0/3284974785' entity='mgr.smithi064.vriyor' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-20T13:01:37.318 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 13:01:37 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T13:01:36.152790+0000 mon.smithi064 (mon.0) 298 : audit [DBG] from='mgr.14156 172.21.15.64:0/3284974785' entity='mgr.smithi064.vriyor' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-20T13:01:37.318 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 13:01:37 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T13:01:36.153813+0000 mon.smithi064 (mon.0) 299 : audit [INF] from='mgr.14156 172.21.15.64:0/3284974785' entity='mgr.smithi064.vriyor' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-20T13:01:37.318 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 13:01:37 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T13:01:36.400902+0000 mon.smithi064 (mon.0) 300 : audit [INF] from='mgr.14156 172.21.15.64:0/3284974785' entity='mgr.smithi064.vriyor' 2024-02-20T13:01:37.319 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 13:01:37 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T13:01:36.767196+0000 mon.smithi064 (mon.0) 301 : audit [INF] from='mgr.14156 172.21.15.64:0/3284974785' entity='mgr.smithi064.vriyor' 2024-02-20T13:01:38.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 13:01:38 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T13:01:37.039744+0000 mon.smithi064 (mon.0) 302 : audit [INF] from='mgr.14156 172.21.15.64:0/3284974785' entity='mgr.smithi064.vriyor' 2024-02-20T13:01:38.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 13:01:38 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T13:01:37.204371+0000 mon.smithi064 (mon.0) 303 : audit [DBG] from='client.? 172.21.15.71:0/2486274179' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-20T13:01:39.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 13:01:39 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T13:01:37.810011+0000 mgr.smithi064.vriyor (mgr.14156) 335 : cluster [DBG] pgmap v316: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T13:01:39.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 13:01:39 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T13:01:38.853051+0000 mon.smithi064 (mon.0) 304 : audit [INF] from='mgr.14156 172.21.15.64:0/3284974785' entity='mgr.smithi064.vriyor' 2024-02-20T13:01:39.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 13:01:39 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T13:01:38.869277+0000 mon.smithi064 (mon.0) 305 : audit [INF] from='mgr.14156 172.21.15.64:0/3284974785' entity='mgr.smithi064.vriyor' 2024-02-20T13:01:39.428 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-20T13:01:39.428 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/daemon-base:latest-pacific shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 680ee126-cfee-11ee-95bb-87774f69a715 -- ceph mon dump -f json 2024-02-20T13:01:40.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 13:01:40 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T13:01:39.111946+0000 mon.smithi064 (mon.0) 306 : audit [INF] from='mgr.14156 172.21.15.64:0/3284974785' entity='mgr.smithi064.vriyor' cmd=[{"prefix": "config rm", "who": "osd/host:smithi064", "name": "osd_memory_target"}]: dispatch 2024-02-20T13:01:40.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 13:01:40 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cephadm 2024-02-20T13:01:39.112504+0000 mgr.smithi064.vriyor (mgr.14156) 336 : cephadm [INF] Updating smithi064:/etc/ceph/ceph.conf 2024-02-20T13:01:40.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 13:01:40 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cephadm 2024-02-20T13:01:39.140806+0000 mgr.smithi064.vriyor (mgr.14156) 337 : cephadm [INF] Updating smithi064:/etc/ceph/ceph.client.admin.keyring 2024-02-20T13:01:40.313 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 13:01:40 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T13:01:39.168258+0000 mon.smithi064 (mon.0) 307 : audit [INF] from='mgr.14156 172.21.15.64:0/3284974785' entity='mgr.smithi064.vriyor' 2024-02-20T13:01:41.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 13:01:41 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T13:01:39.810305+0000 mgr.smithi064.vriyor (mgr.14156) 338 : cluster [DBG] pgmap v317: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T13:01:42.258 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:42.256+0000 7f603fe92700 1 -- 172.21.15.71:0/2232171787 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f60380f5480 msgr2=0x7f60380f58a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T13:01:42.258 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:42.256+0000 7f603fe92700 1 --2- 172.21.15.71:0/2232171787 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f60380f5480 0x7f60380f58a0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 rx=0x7f6028009fb0 tx=0x7f6028009b30).stop 2024-02-20T13:01:42.258 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:42.256+0000 7f603fe92700 1 -- 172.21.15.71:0/2232171787 shutdown_connections 2024-02-20T13:01:42.259 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:42.256+0000 7f603fe92700 1 --2- 172.21.15.71:0/2232171787 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f60380f5480 0x7f60380f58a0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T13:01:42.259 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:42.256+0000 7f603fe92700 1 -- 172.21.15.71:0/2232171787 >> 172.21.15.71:0/2232171787 conn(0x7f60380f0be0 msgr2=0x7f60380f3000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T13:01:42.259 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:42.256+0000 7f603fe92700 1 -- 172.21.15.71:0/2232171787 shutdown_connections 2024-02-20T13:01:42.259 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:42.256+0000 7f603fe92700 1 -- 172.21.15.71:0/2232171787 wait complete. 2024-02-20T13:01:42.259 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:42.257+0000 7f603fe92700 1 Processor -- start 2024-02-20T13:01:42.259 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:42.257+0000 7f603fe92700 1 -- start start 2024-02-20T13:01:42.259 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:42.257+0000 7f603fe92700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f60380f5480 0x7f60381094f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T13:01:42.259 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:42.257+0000 7f603fe92700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6038060320 con 0x7f60380f5480 2024-02-20T13:01:42.259 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:42.257+0000 7f603dc2e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f60380f5480 0x7f60381094f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T13:01:42.259 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:42.257+0000 7f603dc2e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f60380f5480 0x7f60381094f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.71:57380/0 (socket says 172.21.15.71:57380) 2024-02-20T13:01:42.259 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:42.257+0000 7f603dc2e700 1 -- 172.21.15.71:0/3564832239 learned_addr learned my addr 172.21.15.71:0/3564832239 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-02-20T13:01:42.259 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:42.257+0000 7f603dc2e700 1 -- 172.21.15.71:0/3564832239 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6028005040 con 0x7f60380f5480 2024-02-20T13:01:42.259 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:42.257+0000 7f603dc2e700 1 --2- 172.21.15.71:0/3564832239 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f60380f5480 0x7f60381094f0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 rx=0x7f60280089f0 tx=0x7f602800b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T13:01:42.259 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:42.257+0000 7f602e7fc700 1 -- 172.21.15.71:0/3564832239 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6028009320 con 0x7f60380f5480 2024-02-20T13:01:42.260 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:42.257+0000 7f603fe92700 1 -- 172.21.15.71:0/3564832239 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6038060520 con 0x7f60380f5480 2024-02-20T13:01:42.260 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:42.257+0000 7f602e7fc700 1 -- 172.21.15.71:0/3564832239 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(25 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f6028013ec0 con 0x7f60380f5480 2024-02-20T13:01:42.260 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:42.257+0000 7f602e7fc700 1 -- 172.21.15.71:0/3564832239 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6028014d30 con 0x7f60380f5480 2024-02-20T13:01:42.260 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:42.258+0000 7f603fe92700 1 -- 172.21.15.71:0/3564832239 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6038109ba0 con 0x7f60380f5480 2024-02-20T13:01:42.261 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:42.258+0000 7f602e7fc700 1 -- 172.21.15.71:0/3564832239 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 37151+0+0 (secure 0 0 0) 0x7f60280263a0 con 0x7f60380f5480 2024-02-20T13:01:42.261 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:42.258+0000 7f602e7fc700 1 --2- 172.21.15.71:0/3564832239 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f6024031090 0x7f6024033550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T13:01:42.261 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:42.258+0000 7f602e7fc700 1 -- 172.21.15.71:0/3564832239 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1065+0+0 (secure 0 0 0) 0x7f6028042450 con 0x7f60380f5480 2024-02-20T13:01:42.261 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:42.258+0000 7f603fe92700 1 -- 172.21.15.71:0/3564832239 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6038045bb0 con 0x7f60380f5480 2024-02-20T13:01:42.261 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:42.259+0000 7f603d42d700 1 --2- 172.21.15.71:0/3564832239 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f6024031090 0x7f6024033550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T13:01:42.261 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:42.259+0000 7f603d42d700 1 --2- 172.21.15.71:0/3564832239 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f6024031090 0x7f6024033550 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 rx=0x7f60340096a0 tx=0x7f6034006b40).ready entity=mgr.14156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T13:01:42.263 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:42.263+0000 7f602e7fc700 1 -- 172.21.15.71:0/3564832239 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+153237 (secure 0 0 0) 0x7f6028012920 con 0x7f60380f5480 2024-02-20T13:01:42.471 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:42.470+0000 7f603fe92700 1 -- 172.21.15.71:0/3564832239 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f60380fb060 con 0x7f60380f5480 2024-02-20T13:01:42.471 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:42.471+0000 7f602e7fc700 1 -- 172.21.15.71:0/3564832239 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7f602802b020 con 0x7f60380f5480 2024-02-20T13:01:42.472 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-20T13:01:42.472 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":1,"fsid":"680ee126-cfee-11ee-95bb-87774f69a715","modified":"2024-02-20T12:49:44.699059Z","created":"2024-02-20T12:49:44.699059Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi064","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:3300","nonce":0},{"type":"v1","addr":"172.21.15.64:6789","nonce":0}]},"addr":"172.21.15.64:6789/0","public_addr":"172.21.15.64:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-20T13:01:42.473 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:42.472+0000 7f603fe92700 1 -- 172.21.15.71:0/3564832239 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f6024031090 msgr2=0x7f6024033550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T13:01:42.473 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:42.472+0000 7f603fe92700 1 --2- 172.21.15.71:0/3564832239 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f6024031090 0x7f6024033550 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 rx=0x7f60340096a0 tx=0x7f6034006b40).stop 2024-02-20T13:01:42.473 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:42.472+0000 7f603fe92700 1 -- 172.21.15.71:0/3564832239 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f60380f5480 msgr2=0x7f60381094f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T13:01:42.473 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:42.472+0000 7f603fe92700 1 --2- 172.21.15.71:0/3564832239 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f60380f5480 0x7f60381094f0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 rx=0x7f60280089f0 tx=0x7f602800b040).stop 2024-02-20T13:01:42.473 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:42.473+0000 7f603fe92700 1 -- 172.21.15.71:0/3564832239 shutdown_connections 2024-02-20T13:01:42.473 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:42.473+0000 7f603fe92700 1 --2- 172.21.15.71:0/3564832239 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f6024031090 0x7f6024033550 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T13:01:42.473 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:42.473+0000 7f603fe92700 1 --2- 172.21.15.71:0/3564832239 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f60380f5480 0x7f60381094f0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T13:01:42.474 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:42.473+0000 7f603fe92700 1 -- 172.21.15.71:0/3564832239 >> 172.21.15.71:0/3564832239 conn(0x7f60380f0be0 msgr2=0x7f60380f16d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T13:01:42.474 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:42.473+0000 7f603fe92700 1 -- 172.21.15.71:0/3564832239 shutdown_connections 2024-02-20T13:01:42.474 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:42.473+0000 7f603fe92700 1 -- 172.21.15.71:0/3564832239 wait complete. 2024-02-20T13:01:42.474 INFO:teuthology.orchestra.run.smithi071.stderr:dumped monmap epoch 1 2024-02-20T13:01:43.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 13:01:42 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T13:01:41.810663+0000 mgr.smithi064.vriyor (mgr.14156) 339 : cluster [DBG] pgmap v318: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T13:01:43.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 13:01:42 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T13:01:41.825197+0000 mon.smithi064 (mon.0) 308 : audit [INF] from='mgr.14156 172.21.15.64:0/3284974785' entity='mgr.smithi064.vriyor' 2024-02-20T13:01:43.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 13:01:42 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T13:01:42.097680+0000 mon.smithi064 (mon.0) 309 : audit [INF] from='mgr.14156 172.21.15.64:0/3284974785' entity='mgr.smithi064.vriyor' 2024-02-20T13:01:43.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 13:01:42 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T13:01:42.098449+0000 mon.smithi064 (mon.0) 310 : audit [INF] from='mgr.14156 172.21.15.64:0/3284974785' entity='mgr.smithi064.vriyor' cmd=[{"prefix": "config rm", "who": "osd/host:smithi071", "name": "osd_memory_target"}]: dispatch 2024-02-20T13:01:43.313 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 13:01:42 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T13:01:42.099646+0000 mgr.smithi064.vriyor (mgr.14156) 340 : cluster [DBG] pgmap v319: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T13:01:43.313 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 13:01:42 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T13:01:42.104673+0000 mon.smithi064 (mon.0) 311 : audit [INF] from='mgr.14156 172.21.15.64:0/3284974785' entity='mgr.smithi064.vriyor' 2024-02-20T13:01:43.313 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 13:01:42 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cephadm 2024-02-20T13:01:42.107641+0000 mgr.smithi064.vriyor (mgr.14156) 341 : cephadm [INF] Deploying daemon alertmanager.smithi064 on smithi064 2024-02-20T13:01:43.313 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 13:01:42 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T13:01:42.168506+0000 mon.smithi064 (mon.0) 312 : cluster [WRN] Health check failed: 1 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON) 2024-02-20T13:01:43.313 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 13:01:42 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T13:01:42.471788+0000 mon.smithi064 (mon.0) 313 : audit [DBG] from='client.? 172.21.15.71:0/3564832239' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-20T13:01:44.272 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-20T13:01:44.273 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/daemon-base:latest-pacific shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 680ee126-cfee-11ee-95bb-87774f69a715 -- ceph mon dump -f json 2024-02-20T13:01:45.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 13:01:45 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T13:01:44.099930+0000 mgr.smithi064.vriyor (mgr.14156) 342 : cluster [DBG] pgmap v320: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T13:01:46.324 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:46.322+0000 7ff085073700 1 -- 172.21.15.71:0/3827913325 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff0800f5480 msgr2=0x7ff0800f58a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T13:01:46.324 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:46.322+0000 7ff085073700 1 --2- 172.21.15.71:0/3827913325 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff0800f5480 0x7ff0800f58a0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 rx=0x7ff074009fb0 tx=0x7ff074009b30).stop 2024-02-20T13:01:46.324 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:46.322+0000 7ff085073700 1 -- 172.21.15.71:0/3827913325 shutdown_connections 2024-02-20T13:01:46.324 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:46.322+0000 7ff085073700 1 --2- 172.21.15.71:0/3827913325 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff0800f5480 0x7ff0800f58a0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T13:01:46.324 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:46.322+0000 7ff085073700 1 -- 172.21.15.71:0/3827913325 >> 172.21.15.71:0/3827913325 conn(0x7ff0800f0be0 msgr2=0x7ff0800f3000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T13:01:46.325 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:46.322+0000 7ff085073700 1 -- 172.21.15.71:0/3827913325 shutdown_connections 2024-02-20T13:01:46.325 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:46.322+0000 7ff085073700 1 -- 172.21.15.71:0/3827913325 wait complete. 2024-02-20T13:01:46.325 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:46.323+0000 7ff085073700 1 Processor -- start 2024-02-20T13:01:46.325 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:46.323+0000 7ff085073700 1 -- start start 2024-02-20T13:01:46.325 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:46.323+0000 7ff085073700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff0800f5480 0x7ff080109510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T13:01:46.325 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:46.323+0000 7ff085073700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff080060300 con 0x7ff0800f5480 2024-02-20T13:01:46.325 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:46.323+0000 7ff07ed9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff0800f5480 0x7ff080109510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T13:01:46.325 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:46.323+0000 7ff07ed9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff0800f5480 0x7ff080109510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.71:57400/0 (socket says 172.21.15.71:57400) 2024-02-20T13:01:46.325 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:46.323+0000 7ff07ed9d700 1 -- 172.21.15.71:0/2012634789 learned_addr learned my addr 172.21.15.71:0/2012634789 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-02-20T13:01:46.325 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:46.323+0000 7ff07ed9d700 1 -- 172.21.15.71:0/2012634789 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff074005040 con 0x7ff0800f5480 2024-02-20T13:01:46.325 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:46.323+0000 7ff07ed9d700 1 --2- 172.21.15.71:0/2012634789 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff0800f5480 0x7ff080109510 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 rx=0x7ff0740089f0 tx=0x7ff07400b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T13:01:46.325 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:46.324+0000 7ff0677fe700 1 -- 172.21.15.71:0/2012634789 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff074009320 con 0x7ff0800f5480 2024-02-20T13:01:46.325 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:46.324+0000 7ff085073700 1 -- 172.21.15.71:0/2012634789 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff080060500 con 0x7ff0800f5480 2024-02-20T13:01:46.325 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:46.324+0000 7ff0677fe700 1 -- 172.21.15.71:0/2012634789 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(25 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7ff074013ec0 con 0x7ff0800f5480 2024-02-20T13:01:46.325 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:46.324+0000 7ff0677fe700 1 -- 172.21.15.71:0/2012634789 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff074014d30 con 0x7ff0800f5480 2024-02-20T13:01:46.326 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:46.324+0000 7ff085073700 1 -- 172.21.15.71:0/2012634789 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff080109bc0 con 0x7ff0800f5480 2024-02-20T13:01:46.326 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:46.324+0000 7ff0677fe700 1 -- 172.21.15.71:0/2012634789 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 37151+0+0 (secure 0 0 0) 0x7ff0740263a0 con 0x7ff0800f5480 2024-02-20T13:01:46.326 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:46.325+0000 7ff0677fe700 1 --2- 172.21.15.71:0/2012634789 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7ff068031090 0x7ff068033550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T13:01:46.327 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:46.325+0000 7ff0677fe700 1 -- 172.21.15.71:0/2012634789 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1065+0+0 (secure 0 0 0) 0x7ff0740423a0 con 0x7ff0800f5480 2024-02-20T13:01:46.327 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:46.325+0000 7ff085073700 1 -- 172.21.15.71:0/2012634789 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0800591e0 con 0x7ff0800f5480 2024-02-20T13:01:46.327 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:46.325+0000 7ff07e59c700 1 --2- 172.21.15.71:0/2012634789 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7ff068031090 0x7ff068033550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T13:01:46.327 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:46.325+0000 7ff07e59c700 1 --2- 172.21.15.71:0/2012634789 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7ff068031090 0x7ff068033550 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 rx=0x7ff0700096a0 tx=0x7ff070006b40).ready entity=mgr.14156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T13:01:46.330 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:46.330+0000 7ff0677fe700 1 -- 172.21.15.71:0/2012634789 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+153237 (secure 0 0 0) 0x7ff0740266d0 con 0x7ff0800f5480 2024-02-20T13:01:46.538 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:46.537+0000 7ff085073700 1 -- 172.21.15.71:0/2012634789 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7ff0800faf70 con 0x7ff0800f5480 2024-02-20T13:01:46.539 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:46.538+0000 7ff0677fe700 1 -- 172.21.15.71:0/2012634789 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7ff074012330 con 0x7ff0800f5480 2024-02-20T13:01:46.539 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-20T13:01:46.539 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":1,"fsid":"680ee126-cfee-11ee-95bb-87774f69a715","modified":"2024-02-20T12:49:44.699059Z","created":"2024-02-20T12:49:44.699059Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi064","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:3300","nonce":0},{"type":"v1","addr":"172.21.15.64:6789","nonce":0}]},"addr":"172.21.15.64:6789/0","public_addr":"172.21.15.64:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-20T13:01:46.540 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:46.540+0000 7ff085073700 1 -- 172.21.15.71:0/2012634789 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7ff068031090 msgr2=0x7ff068033550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T13:01:46.541 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:46.540+0000 7ff085073700 1 --2- 172.21.15.71:0/2012634789 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7ff068031090 0x7ff068033550 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 rx=0x7ff0700096a0 tx=0x7ff070006b40).stop 2024-02-20T13:01:46.541 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:46.540+0000 7ff085073700 1 -- 172.21.15.71:0/2012634789 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff0800f5480 msgr2=0x7ff080109510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T13:01:46.541 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:46.540+0000 7ff085073700 1 --2- 172.21.15.71:0/2012634789 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff0800f5480 0x7ff080109510 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 rx=0x7ff0740089f0 tx=0x7ff07400b040).stop 2024-02-20T13:01:46.541 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:46.540+0000 7ff085073700 1 -- 172.21.15.71:0/2012634789 shutdown_connections 2024-02-20T13:01:46.541 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:46.540+0000 7ff085073700 1 --2- 172.21.15.71:0/2012634789 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7ff068031090 0x7ff068033550 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T13:01:46.541 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:46.540+0000 7ff085073700 1 --2- 172.21.15.71:0/2012634789 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff0800f5480 0x7ff080109510 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T13:01:46.541 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:46.540+0000 7ff085073700 1 -- 172.21.15.71:0/2012634789 >> 172.21.15.71:0/2012634789 conn(0x7ff0800f0be0 msgr2=0x7ff0800f16d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T13:01:46.541 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:46.540+0000 7ff085073700 1 -- 172.21.15.71:0/2012634789 shutdown_connections 2024-02-20T13:01:46.541 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:46.540+0000 7ff085073700 1 -- 172.21.15.71:0/2012634789 wait complete. 2024-02-20T13:01:46.542 INFO:teuthology.orchestra.run.smithi071.stderr:dumped monmap epoch 1 2024-02-20T13:01:47.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 13:01:47 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T13:01:46.100285+0000 mgr.smithi064.vriyor (mgr.14156) 343 : cluster [DBG] pgmap v321: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T13:01:47.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 13:01:47 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T13:01:46.538847+0000 mon.smithi064 (mon.0) 314 : audit [DBG] from='client.? 172.21.15.71:0/2012634789' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-20T13:01:48.238 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-20T13:01:48.238 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/daemon-base:latest-pacific shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 680ee126-cfee-11ee-95bb-87774f69a715 -- ceph mon dump -f json 2024-02-20T13:01:49.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 13:01:49 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T13:01:48.100646+0000 mgr.smithi064.vriyor (mgr.14156) 344 : cluster [DBG] pgmap v322: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T13:01:50.159 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:50.158+0000 7f8e3961b700 1 -- 172.21.15.71:0/3746520775 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8e340f5480 msgr2=0x7f8e340f58a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T13:01:50.159 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:50.158+0000 7f8e3961b700 1 --2- 172.21.15.71:0/3746520775 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8e340f5480 0x7f8e340f58a0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 rx=0x7f8e1c009fb0 tx=0x7f8e1c009b30).stop 2024-02-20T13:01:50.159 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:50.158+0000 7f8e3961b700 1 -- 172.21.15.71:0/3746520775 shutdown_connections 2024-02-20T13:01:50.159 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:50.158+0000 7f8e3961b700 1 --2- 172.21.15.71:0/3746520775 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8e340f5480 0x7f8e340f58a0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T13:01:50.159 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:50.158+0000 7f8e3961b700 1 -- 172.21.15.71:0/3746520775 >> 172.21.15.71:0/3746520775 conn(0x7f8e340f0be0 msgr2=0x7f8e340f3000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T13:01:50.159 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:50.158+0000 7f8e3961b700 1 -- 172.21.15.71:0/3746520775 shutdown_connections 2024-02-20T13:01:50.159 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:50.158+0000 7f8e3961b700 1 -- 172.21.15.71:0/3746520775 wait complete. 2024-02-20T13:01:50.161 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:50.159+0000 7f8e3961b700 1 Processor -- start 2024-02-20T13:01:50.161 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:50.159+0000 7f8e3961b700 1 -- start start 2024-02-20T13:01:50.161 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:50.159+0000 7f8e3961b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8e340f5480 0x7f8e341067f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T13:01:50.162 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:50.159+0000 7f8e3961b700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e34106ce0 con 0x7f8e340f5480 2024-02-20T13:01:50.162 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:50.160+0000 7f8e32ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8e340f5480 0x7f8e341067f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T13:01:50.162 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:50.160+0000 7f8e32ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8e340f5480 0x7f8e341067f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.71:42144/0 (socket says 172.21.15.71:42144) 2024-02-20T13:01:50.162 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:50.160+0000 7f8e32ffd700 1 -- 172.21.15.71:0/2083447320 learned_addr learned my addr 172.21.15.71:0/2083447320 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-02-20T13:01:50.162 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:50.160+0000 7f8e32ffd700 1 -- 172.21.15.71:0/2083447320 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e1c005040 con 0x7f8e340f5480 2024-02-20T13:01:50.162 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:50.160+0000 7f8e32ffd700 1 --2- 172.21.15.71:0/2083447320 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8e340f5480 0x7f8e341067f0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 rx=0x7f8e1c000c00 tx=0x7f8e1c00b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T13:01:50.162 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:50.160+0000 7f8e2b7fe700 1 -- 172.21.15.71:0/2083447320 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8e1c0095f0 con 0x7f8e340f5480 2024-02-20T13:01:50.162 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:50.160+0000 7f8e3961b700 1 -- 172.21.15.71:0/2083447320 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e34106ee0 con 0x7f8e340f5480 2024-02-20T13:01:50.162 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:50.160+0000 7f8e2b7fe700 1 -- 172.21.15.71:0/2083447320 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(25 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f8e1c007e50 con 0x7f8e340f5480 2024-02-20T13:01:50.162 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:50.160+0000 7f8e2b7fe700 1 -- 172.21.15.71:0/2083447320 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8e1c014e70 con 0x7f8e340f5480 2024-02-20T13:01:50.162 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:50.160+0000 7f8e3961b700 1 -- 172.21.15.71:0/2083447320 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e34107370 con 0x7f8e340f5480 2024-02-20T13:01:50.162 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:50.161+0000 7f8e2b7fe700 1 -- 172.21.15.71:0/2083447320 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 37151+0+0 (secure 0 0 0) 0x7f8e1c0079c0 con 0x7f8e340f5480 2024-02-20T13:01:50.162 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:50.161+0000 7f8e2b7fe700 1 --2- 172.21.15.71:0/2083447320 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f8e20031090 0x7f8e20033550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T13:01:50.163 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:50.161+0000 7f8e2b7fe700 1 -- 172.21.15.71:0/2083447320 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1065+0+0 (secure 0 0 0) 0x7f8e1c041c50 con 0x7f8e340f5480 2024-02-20T13:01:50.163 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:50.161+0000 7f8e3961b700 1 -- 172.21.15.71:0/2083447320 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e34045bb0 con 0x7f8e340f5480 2024-02-20T13:01:50.163 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:50.161+0000 7f8e327fc700 1 --2- 172.21.15.71:0/2083447320 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f8e20031090 0x7f8e20033550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T13:01:50.163 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:50.162+0000 7f8e327fc700 1 --2- 172.21.15.71:0/2083447320 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f8e20031090 0x7f8e20033550 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 rx=0x7f8e240096a0 tx=0x7f8e24006b40).ready entity=mgr.14156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T13:01:50.166 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:50.166+0000 7f8e2b7fe700 1 -- 172.21.15.71:0/2083447320 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+153237 (secure 0 0 0) 0x7f8e1c01b910 con 0x7f8e340f5480 2024-02-20T13:01:50.374 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:50.373+0000 7f8e3961b700 1 -- 172.21.15.71:0/2083447320 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f8e340fb060 con 0x7f8e340f5480 2024-02-20T13:01:50.374 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:50.374+0000 7f8e2b7fe700 1 -- 172.21.15.71:0/2083447320 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7f8e1c02c090 con 0x7f8e340f5480 2024-02-20T13:01:50.374 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-20T13:01:50.374 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":1,"fsid":"680ee126-cfee-11ee-95bb-87774f69a715","modified":"2024-02-20T12:49:44.699059Z","created":"2024-02-20T12:49:44.699059Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi064","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:3300","nonce":0},{"type":"v1","addr":"172.21.15.64:6789","nonce":0}]},"addr":"172.21.15.64:6789/0","public_addr":"172.21.15.64:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-20T13:01:50.376 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:50.375+0000 7f8e3961b700 1 -- 172.21.15.71:0/2083447320 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f8e20031090 msgr2=0x7f8e20033550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T13:01:50.376 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:50.375+0000 7f8e3961b700 1 --2- 172.21.15.71:0/2083447320 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f8e20031090 0x7f8e20033550 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 rx=0x7f8e240096a0 tx=0x7f8e24006b40).stop 2024-02-20T13:01:50.376 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:50.375+0000 7f8e3961b700 1 -- 172.21.15.71:0/2083447320 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8e340f5480 msgr2=0x7f8e341067f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T13:01:50.376 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:50.375+0000 7f8e3961b700 1 --2- 172.21.15.71:0/2083447320 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8e340f5480 0x7f8e341067f0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 rx=0x7f8e1c000c00 tx=0x7f8e1c00b040).stop 2024-02-20T13:01:50.376 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:50.375+0000 7f8e3961b700 1 -- 172.21.15.71:0/2083447320 shutdown_connections 2024-02-20T13:01:50.376 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:50.375+0000 7f8e3961b700 1 --2- 172.21.15.71:0/2083447320 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f8e20031090 0x7f8e20033550 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T13:01:50.377 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:50.375+0000 7f8e3961b700 1 --2- 172.21.15.71:0/2083447320 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8e340f5480 0x7f8e341067f0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T13:01:50.377 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:50.375+0000 7f8e3961b700 1 -- 172.21.15.71:0/2083447320 >> 172.21.15.71:0/2083447320 conn(0x7f8e340f0be0 msgr2=0x7f8e340f16d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T13:01:50.377 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:50.375+0000 7f8e3961b700 1 -- 172.21.15.71:0/2083447320 shutdown_connections 2024-02-20T13:01:50.377 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:50.375+0000 7f8e3961b700 1 -- 172.21.15.71:0/2083447320 wait complete. 2024-02-20T13:01:50.377 INFO:teuthology.orchestra.run.smithi071.stderr:dumped monmap epoch 1 2024-02-20T13:01:51.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 13:01:51 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T13:01:50.101033+0000 mgr.smithi064.vriyor (mgr.14156) 345 : cluster [DBG] pgmap v323: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T13:01:51.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 13:01:51 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T13:01:50.374521+0000 mon.smithi064 (mon.0) 315 : audit [DBG] from='client.? 172.21.15.71:0/2083447320' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-20T13:01:52.152 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-20T13:01:52.152 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/daemon-base:latest-pacific shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 680ee126-cfee-11ee-95bb-87774f69a715 -- ceph mon dump -f json 2024-02-20T13:01:53.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 13:01:53 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T13:01:52.101260+0000 mgr.smithi064.vriyor (mgr.14156) 346 : cluster [DBG] pgmap v324: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T13:01:54.077 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:54.075+0000 7fd2d0b0a700 1 -- 172.21.15.71:0/2641247501 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd2cc0f6590 msgr2=0x7fd2cc0f69b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T13:01:54.077 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:54.075+0000 7fd2d0b0a700 1 --2- 172.21.15.71:0/2641247501 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd2cc0f6590 0x7fd2cc0f69b0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 rx=0x7fd2c0004760 tx=0x7fd2c0009ad0).stop 2024-02-20T13:01:54.077 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:54.075+0000 7fd2d0b0a700 1 -- 172.21.15.71:0/2641247501 shutdown_connections 2024-02-20T13:01:54.077 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:54.075+0000 7fd2d0b0a700 1 --2- 172.21.15.71:0/2641247501 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd2cc0f6590 0x7fd2cc0f69b0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T13:01:54.077 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:54.075+0000 7fd2d0b0a700 1 -- 172.21.15.71:0/2641247501 >> 172.21.15.71:0/2641247501 conn(0x7fd2cc0f1cd0 msgr2=0x7fd2cc0f4110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T13:01:54.077 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:54.075+0000 7fd2d0b0a700 1 -- 172.21.15.71:0/2641247501 shutdown_connections 2024-02-20T13:01:54.078 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:54.075+0000 7fd2d0b0a700 1 -- 172.21.15.71:0/2641247501 wait complete. 2024-02-20T13:01:54.078 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:54.076+0000 7fd2d0b0a700 1 Processor -- start 2024-02-20T13:01:54.078 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:54.076+0000 7fd2d0b0a700 1 -- start start 2024-02-20T13:01:54.078 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:54.076+0000 7fd2d0b0a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd2cc0f6590 0x7fd2cc1067c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T13:01:54.078 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:54.076+0000 7fd2d0b0a700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2cc106cb0 con 0x7fd2cc0f6590 2024-02-20T13:01:54.078 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:54.076+0000 7fd2ca59c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd2cc0f6590 0x7fd2cc1067c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T13:01:54.078 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:54.076+0000 7fd2ca59c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd2cc0f6590 0x7fd2cc1067c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.71:42152/0 (socket says 172.21.15.71:42152) 2024-02-20T13:01:54.078 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:54.076+0000 7fd2ca59c700 1 -- 172.21.15.71:0/1456579751 learned_addr learned my addr 172.21.15.71:0/1456579751 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-02-20T13:01:54.078 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:54.076+0000 7fd2ca59c700 1 -- 172.21.15.71:0/1456579751 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2c0009d70 con 0x7fd2cc0f6590 2024-02-20T13:01:54.078 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:54.076+0000 7fd2ca59c700 1 --2- 172.21.15.71:0/1456579751 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd2cc0f6590 0x7fd2cc1067c0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 rx=0x7fd2c0004170 tx=0x7fd2c0009ad0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T13:01:54.078 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:54.076+0000 7fd2baffd700 1 -- 172.21.15.71:0/1456579751 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd2c000e040 con 0x7fd2cc0f6590 2024-02-20T13:01:54.078 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:54.077+0000 7fd2baffd700 1 -- 172.21.15.71:0/1456579751 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(25 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7fd2c0008ef0 con 0x7fd2cc0f6590 2024-02-20T13:01:54.078 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:54.077+0000 7fd2d0b0a700 1 -- 172.21.15.71:0/1456579751 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2cc106eb0 con 0x7fd2cc0f6590 2024-02-20T13:01:54.079 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:54.077+0000 7fd2baffd700 1 -- 172.21.15.71:0/1456579751 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd2c0014c90 con 0x7fd2cc0f6590 2024-02-20T13:01:54.079 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:54.077+0000 7fd2d0b0a700 1 -- 172.21.15.71:0/1456579751 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2cc107340 con 0x7fd2cc0f6590 2024-02-20T13:01:54.080 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:54.077+0000 7fd2baffd700 1 -- 172.21.15.71:0/1456579751 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 37151+0+0 (secure 0 0 0) 0x7fd2c0014420 con 0x7fd2cc0f6590 2024-02-20T13:01:54.080 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:54.078+0000 7fd2d0b0a700 1 -- 172.21.15.71:0/1456579751 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2cc045bb0 con 0x7fd2cc0f6590 2024-02-20T13:01:54.080 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:54.078+0000 7fd2baffd700 1 --2- 172.21.15.71:0/1456579751 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fd2b4031090 0x7fd2b4033550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T13:01:54.080 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:54.078+0000 7fd2baffd700 1 -- 172.21.15.71:0/1456579751 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1065+0+0 (secure 0 0 0) 0x7fd2c0042240 con 0x7fd2cc0f6590 2024-02-20T13:01:54.080 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:54.078+0000 7fd2c9d9b700 1 --2- 172.21.15.71:0/1456579751 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fd2b4031090 0x7fd2b4033550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T13:01:54.080 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:54.078+0000 7fd2c9d9b700 1 --2- 172.21.15.71:0/1456579751 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fd2b4031090 0x7fd2b4033550 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 rx=0x7fd2bc0096a0 tx=0x7fd2bc006b40).ready entity=mgr.14156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T13:01:54.083 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:54.082+0000 7fd2baffd700 1 -- 172.21.15.71:0/1456579751 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+153237 (secure 0 0 0) 0x7fd2c0026300 con 0x7fd2cc0f6590 2024-02-20T13:01:54.289 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:54.289+0000 7fd2d0b0a700 1 -- 172.21.15.71:0/1456579751 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fd2cc1081f0 con 0x7fd2cc0f6590 2024-02-20T13:01:54.290 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:54.290+0000 7fd2baffd700 1 -- 172.21.15.71:0/1456579751 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7fd2c0012b20 con 0x7fd2cc0f6590 2024-02-20T13:01:54.290 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-20T13:01:54.291 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":1,"fsid":"680ee126-cfee-11ee-95bb-87774f69a715","modified":"2024-02-20T12:49:44.699059Z","created":"2024-02-20T12:49:44.699059Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi064","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:3300","nonce":0},{"type":"v1","addr":"172.21.15.64:6789","nonce":0}]},"addr":"172.21.15.64:6789/0","public_addr":"172.21.15.64:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-20T13:01:54.292 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:54.291+0000 7fd2d0b0a700 1 -- 172.21.15.71:0/1456579751 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fd2b4031090 msgr2=0x7fd2b4033550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T13:01:54.292 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:54.291+0000 7fd2d0b0a700 1 --2- 172.21.15.71:0/1456579751 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fd2b4031090 0x7fd2b4033550 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 rx=0x7fd2bc0096a0 tx=0x7fd2bc006b40).stop 2024-02-20T13:01:54.292 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:54.291+0000 7fd2d0b0a700 1 -- 172.21.15.71:0/1456579751 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd2cc0f6590 msgr2=0x7fd2cc1067c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T13:01:54.292 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:54.291+0000 7fd2d0b0a700 1 --2- 172.21.15.71:0/1456579751 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd2cc0f6590 0x7fd2cc1067c0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 rx=0x7fd2c0004170 tx=0x7fd2c0009ad0).stop 2024-02-20T13:01:54.292 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:54.291+0000 7fd2d0b0a700 1 -- 172.21.15.71:0/1456579751 shutdown_connections 2024-02-20T13:01:54.292 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:54.291+0000 7fd2d0b0a700 1 --2- 172.21.15.71:0/1456579751 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fd2b4031090 0x7fd2b4033550 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T13:01:54.292 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:54.291+0000 7fd2d0b0a700 1 --2- 172.21.15.71:0/1456579751 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd2cc0f6590 0x7fd2cc1067c0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T13:01:54.293 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:54.291+0000 7fd2d0b0a700 1 -- 172.21.15.71:0/1456579751 >> 172.21.15.71:0/1456579751 conn(0x7fd2cc0f1cd0 msgr2=0x7fd2cc0f27c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T13:01:54.293 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:54.291+0000 7fd2d0b0a700 1 -- 172.21.15.71:0/1456579751 shutdown_connections 2024-02-20T13:01:54.293 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:54.291+0000 7fd2d0b0a700 1 -- 172.21.15.71:0/1456579751 wait complete. 2024-02-20T13:01:54.293 INFO:teuthology.orchestra.run.smithi071.stderr:dumped monmap epoch 1 2024-02-20T13:01:54.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 13:01:54 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T13:01:53.854616+0000 mon.smithi064 (mon.0) 316 : audit [INF] from='mgr.14156 172.21.15.64:0/3284974785' entity='mgr.smithi064.vriyor' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi064.vriyor/mirror_snapshot_schedule"}]: dispatch 2024-02-20T13:01:54.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 13:01:54 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T13:01:53.854858+0000 mon.smithi064 (mon.0) 317 : audit [INF] from='mgr.14156 172.21.15.64:0/3284974785' entity='mgr.smithi064.vriyor' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi064.vriyor/trash_purge_schedule"}]: dispatch 2024-02-20T13:01:55.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 13:01:55 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T13:01:54.101521+0000 mgr.smithi064.vriyor (mgr.14156) 347 : cluster [DBG] pgmap v325: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T13:01:55.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 13:01:55 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T13:01:54.290404+0000 mon.smithi064 (mon.0) 318 : audit [DBG] from='client.? 172.21.15.71:0/1456579751' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-20T13:01:55.907 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-20T13:01:55.907 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/daemon-base:latest-pacific shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 680ee126-cfee-11ee-95bb-87774f69a715 -- ceph mon dump -f json 2024-02-20T13:01:57.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 13:01:57 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T13:01:56.101872+0000 mgr.smithi064.vriyor (mgr.14156) 348 : cluster [DBG] pgmap v326: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T13:01:58.148 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:58.145+0000 7f6d0f59e700 1 -- 172.21.15.71:0/2913436605 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6d100f63e0 msgr2=0x7f6d100f6800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T13:01:58.149 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:58.145+0000 7f6d0f59e700 1 --2- 172.21.15.71:0/2913436605 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6d100f63e0 0x7f6d100f6800 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 rx=0x7f6d04009fb0 tx=0x7f6d04009b30).stop 2024-02-20T13:01:58.149 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:58.145+0000 7f6d0f59e700 1 -- 172.21.15.71:0/2913436605 shutdown_connections 2024-02-20T13:01:58.149 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:58.145+0000 7f6d0f59e700 1 --2- 172.21.15.71:0/2913436605 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6d100f63e0 0x7f6d100f6800 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T13:01:58.149 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:58.145+0000 7f6d0f59e700 1 -- 172.21.15.71:0/2913436605 >> 172.21.15.71:0/2913436605 conn(0x7f6d100f1b40 msgr2=0x7f6d100f3f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T13:01:58.149 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:58.145+0000 7f6d0f59e700 1 -- 172.21.15.71:0/2913436605 shutdown_connections 2024-02-20T13:01:58.149 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:58.145+0000 7f6d0f59e700 1 -- 172.21.15.71:0/2913436605 wait complete. 2024-02-20T13:01:58.149 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:58.145+0000 7f6d0f59e700 1 Processor -- start 2024-02-20T13:01:58.149 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:58.145+0000 7f6d0f59e700 1 -- start start 2024-02-20T13:01:58.149 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:58.145+0000 7f6d0f59e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6d100f63e0 0x7f6d10106620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T13:01:58.149 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:58.145+0000 7f6d0f59e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d10106b10 con 0x7f6d100f63e0 2024-02-20T13:01:58.150 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:58.146+0000 7f6d0e59c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6d100f63e0 0x7f6d10106620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T13:01:58.150 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:58.146+0000 7f6d0e59c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6d100f63e0 0x7f6d10106620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.71:36484/0 (socket says 172.21.15.71:36484) 2024-02-20T13:01:58.150 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:58.146+0000 7f6d0e59c700 1 -- 172.21.15.71:0/2752534856 learned_addr learned my addr 172.21.15.71:0/2752534856 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-02-20T13:01:58.150 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:58.146+0000 7f6d0e59c700 1 -- 172.21.15.71:0/2752534856 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d04005040 con 0x7f6d100f63e0 2024-02-20T13:01:58.150 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:58.146+0000 7f6d0e59c700 1 --2- 172.21.15.71:0/2752534856 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6d100f63e0 0x7f6d10106620 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 rx=0x7f6d040089f0 tx=0x7f6d0400b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T13:01:58.150 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:58.146+0000 7f6cfeffd700 1 -- 172.21.15.71:0/2752534856 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6d04009320 con 0x7f6d100f63e0 2024-02-20T13:01:58.150 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:58.146+0000 7f6d0f59e700 1 -- 172.21.15.71:0/2752534856 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d10106d10 con 0x7f6d100f63e0 2024-02-20T13:01:58.150 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:58.146+0000 7f6cfeffd700 1 -- 172.21.15.71:0/2752534856 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(25 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f6d04013ec0 con 0x7f6d100f63e0 2024-02-20T13:01:58.150 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:58.146+0000 7f6cfeffd700 1 -- 172.21.15.71:0/2752534856 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6d04014d30 con 0x7f6d100f63e0 2024-02-20T13:01:58.150 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:58.146+0000 7f6d0f59e700 1 -- 172.21.15.71:0/2752534856 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d101071a0 con 0x7f6d100f63e0 2024-02-20T13:01:58.150 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:58.147+0000 7f6cfeffd700 1 -- 172.21.15.71:0/2752534856 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 37151+0+0 (secure 0 0 0) 0x7f6d040095f0 con 0x7f6d100f63e0 2024-02-20T13:01:58.151 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:58.147+0000 7f6cfeffd700 1 --2- 172.21.15.71:0/2752534856 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f6cf8031090 0x7f6cf8033550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T13:01:58.151 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:58.147+0000 7f6cfeffd700 1 -- 172.21.15.71:0/2752534856 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1065+0+0 (secure 0 0 0) 0x7f6d04042420 con 0x7f6d100f63e0 2024-02-20T13:01:58.151 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:58.147+0000 7f6d0f59e700 1 -- 172.21.15.71:0/2752534856 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d10045bb0 con 0x7f6d100f63e0 2024-02-20T13:01:58.151 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:58.147+0000 7f6d0dd9b700 1 --2- 172.21.15.71:0/2752534856 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f6cf8031090 0x7f6cf8033550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T13:01:58.151 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:58.148+0000 7f6d0dd9b700 1 --2- 172.21.15.71:0/2752534856 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f6cf8031090 0x7f6cf8033550 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 rx=0x7f6d000096a0 tx=0x7f6d00006b40).ready entity=mgr.14156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T13:01:58.152 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:58.152+0000 7f6cfeffd700 1 -- 172.21.15.71:0/2752534856 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+153237 (secure 0 0 0) 0x7f6d0401d7f0 con 0x7f6d100f63e0 2024-02-20T13:01:58.359 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:58.359+0000 7f6d0f59e700 1 -- 172.21.15.71:0/2752534856 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f6d100fbfc0 con 0x7f6d100f63e0 2024-02-20T13:01:58.360 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:58.359+0000 7f6cfeffd700 1 -- 172.21.15.71:0/2752534856 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7f6d0402b020 con 0x7f6d100f63e0 2024-02-20T13:01:58.360 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-20T13:01:58.360 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":1,"fsid":"680ee126-cfee-11ee-95bb-87774f69a715","modified":"2024-02-20T12:49:44.699059Z","created":"2024-02-20T12:49:44.699059Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi064","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:3300","nonce":0},{"type":"v1","addr":"172.21.15.64:6789","nonce":0}]},"addr":"172.21.15.64:6789/0","public_addr":"172.21.15.64:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-20T13:01:58.362 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:58.361+0000 7f6d0f59e700 1 -- 172.21.15.71:0/2752534856 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f6cf8031090 msgr2=0x7f6cf8033550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T13:01:58.362 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:58.361+0000 7f6d0f59e700 1 --2- 172.21.15.71:0/2752534856 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f6cf8031090 0x7f6cf8033550 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 rx=0x7f6d000096a0 tx=0x7f6d00006b40).stop 2024-02-20T13:01:58.362 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:58.361+0000 7f6d0f59e700 1 -- 172.21.15.71:0/2752534856 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6d100f63e0 msgr2=0x7f6d10106620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T13:01:58.362 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:58.361+0000 7f6d0f59e700 1 --2- 172.21.15.71:0/2752534856 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6d100f63e0 0x7f6d10106620 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 rx=0x7f6d040089f0 tx=0x7f6d0400b040).stop 2024-02-20T13:01:58.362 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:58.361+0000 7f6d0f59e700 1 -- 172.21.15.71:0/2752534856 shutdown_connections 2024-02-20T13:01:58.362 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:58.361+0000 7f6d0f59e700 1 --2- 172.21.15.71:0/2752534856 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f6cf8031090 0x7f6cf8033550 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T13:01:58.362 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:58.361+0000 7f6d0f59e700 1 --2- 172.21.15.71:0/2752534856 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6d100f63e0 0x7f6d10106620 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T13:01:58.362 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:58.361+0000 7f6d0f59e700 1 -- 172.21.15.71:0/2752534856 >> 172.21.15.71:0/2752534856 conn(0x7f6d100f1b40 msgr2=0x7f6d100f2660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T13:01:58.362 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:58.361+0000 7f6d0f59e700 1 -- 172.21.15.71:0/2752534856 shutdown_connections 2024-02-20T13:01:58.362 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:01:58.361+0000 7f6d0f59e700 1 -- 172.21.15.71:0/2752534856 wait complete. 2024-02-20T13:01:58.363 INFO:teuthology.orchestra.run.smithi071.stderr:dumped monmap epoch 1 2024-02-20T13:01:59.561 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 13:01:59 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T13:01:58.102251+0000 mgr.smithi064.vriyor (mgr.14156) 349 : cluster [DBG] pgmap v327: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T13:01:59.561 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 13:01:59 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T13:01:58.360295+0000 mon.smithi064 (mon.0) 319 : audit [DBG] from='client.? 172.21.15.71:0/2752534856' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-20T13:02:00.005 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-20T13:02:00.005 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/daemon-base:latest-pacific shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 680ee126-cfee-11ee-95bb-87774f69a715 -- ceph mon dump -f json 2024-02-20T13:02:01.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 13:02:01 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T13:02:00.102737+0000 mgr.smithi064.vriyor (mgr.14156) 350 : cluster [DBG] pgmap v328: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T13:02:01.857 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:01.854+0000 7f0576164700 1 -- 172.21.15.71:0/5877486 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05700f6590 msgr2=0x7f05700f69b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T13:02:01.857 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:01.854+0000 7f0576164700 1 --2- 172.21.15.71:0/5877486 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05700f6590 0x7f05700f69b0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 rx=0x7f0560004760 tx=0x7f0560009ad0).stop 2024-02-20T13:02:01.857 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:01.855+0000 7f0576164700 1 -- 172.21.15.71:0/5877486 shutdown_connections 2024-02-20T13:02:01.857 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:01.855+0000 7f0576164700 1 --2- 172.21.15.71:0/5877486 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05700f6590 0x7f05700f69b0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T13:02:01.857 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:01.855+0000 7f0576164700 1 -- 172.21.15.71:0/5877486 >> 172.21.15.71:0/5877486 conn(0x7f05700f1cd0 msgr2=0x7f05700f4110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T13:02:01.857 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:01.855+0000 7f0576164700 1 -- 172.21.15.71:0/5877486 shutdown_connections 2024-02-20T13:02:01.857 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:01.855+0000 7f0576164700 1 -- 172.21.15.71:0/5877486 wait complete. 2024-02-20T13:02:01.857 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:01.855+0000 7f0576164700 1 Processor -- start 2024-02-20T13:02:01.858 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:01.855+0000 7f0576164700 1 -- start start 2024-02-20T13:02:01.858 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:01.856+0000 7f0576164700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05700f6590 0x7f05701067c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T13:02:01.858 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:01.856+0000 7f0576164700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0570106cb0 con 0x7f05700f6590 2024-02-20T13:02:01.858 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:01.856+0000 7f056f7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05700f6590 0x7f05701067c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T13:02:01.858 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:01.856+0000 7f056f7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05700f6590 0x7f05701067c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.71:36498/0 (socket says 172.21.15.71:36498) 2024-02-20T13:02:01.858 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:01.856+0000 7f056f7fe700 1 -- 172.21.15.71:0/677128477 learned_addr learned my addr 172.21.15.71:0/677128477 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-02-20T13:02:01.858 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:01.856+0000 7f056f7fe700 1 -- 172.21.15.71:0/677128477 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0560009d70 con 0x7f05700f6590 2024-02-20T13:02:01.858 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:01.856+0000 7f056f7fe700 1 --2- 172.21.15.71:0/677128477 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05700f6590 0x7f05701067c0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 rx=0x7f0560004170 tx=0x7f0560009ad0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T13:02:01.858 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:01.856+0000 7f0557fff700 1 -- 172.21.15.71:0/677128477 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f056000e040 con 0x7f05700f6590 2024-02-20T13:02:01.858 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:01.856+0000 7f0576164700 1 -- 172.21.15.71:0/677128477 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0570106eb0 con 0x7f05700f6590 2024-02-20T13:02:01.858 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:01.856+0000 7f0557fff700 1 -- 172.21.15.71:0/677128477 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(25 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f0560008ef0 con 0x7f05700f6590 2024-02-20T13:02:01.858 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:01.856+0000 7f0557fff700 1 -- 172.21.15.71:0/677128477 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0560014c90 con 0x7f05700f6590 2024-02-20T13:02:01.858 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:01.857+0000 7f0576164700 1 -- 172.21.15.71:0/677128477 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0570107340 con 0x7f05700f6590 2024-02-20T13:02:01.859 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:01.857+0000 7f0557fff700 1 -- 172.21.15.71:0/677128477 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 37151+0+0 (secure 0 0 0) 0x7f0560014420 con 0x7f05700f6590 2024-02-20T13:02:01.859 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:01.858+0000 7f0576164700 1 -- 172.21.15.71:0/677128477 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0570045bb0 con 0x7f05700f6590 2024-02-20T13:02:01.859 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:01.858+0000 7f0557fff700 1 --2- 172.21.15.71:0/677128477 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f0558031090 0x7f0558033550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T13:02:01.859 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:01.858+0000 7f0557fff700 1 -- 172.21.15.71:0/677128477 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1065+0+0 (secure 0 0 0) 0x7f05600422c0 con 0x7f05700f6590 2024-02-20T13:02:01.859 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:01.858+0000 7f056effd700 1 --2- 172.21.15.71:0/677128477 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f0558031090 0x7f0558033550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T13:02:01.860 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:01.858+0000 7f056effd700 1 --2- 172.21.15.71:0/677128477 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f0558031090 0x7f0558033550 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 rx=0x7f05640096a0 tx=0x7f0564006b40).ready entity=mgr.14156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T13:02:01.863 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:01.862+0000 7f0557fff700 1 -- 172.21.15.71:0/677128477 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+153237 (secure 0 0 0) 0x7f0560026300 con 0x7f05700f6590 2024-02-20T13:02:02.070 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:02.070+0000 7f0576164700 1 -- 172.21.15.71:0/677128477 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f05701081f0 con 0x7f05700f6590 2024-02-20T13:02:02.071 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:02.071+0000 7f0557fff700 1 -- 172.21.15.71:0/677128477 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7f0560012b20 con 0x7f05700f6590 2024-02-20T13:02:02.071 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-20T13:02:02.071 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":1,"fsid":"680ee126-cfee-11ee-95bb-87774f69a715","modified":"2024-02-20T12:49:44.699059Z","created":"2024-02-20T12:49:44.699059Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi064","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:3300","nonce":0},{"type":"v1","addr":"172.21.15.64:6789","nonce":0}]},"addr":"172.21.15.64:6789/0","public_addr":"172.21.15.64:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-20T13:02:02.073 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:02.072+0000 7f0576164700 1 -- 172.21.15.71:0/677128477 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f0558031090 msgr2=0x7f0558033550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T13:02:02.073 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:02.072+0000 7f0576164700 1 --2- 172.21.15.71:0/677128477 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f0558031090 0x7f0558033550 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 rx=0x7f05640096a0 tx=0x7f0564006b40).stop 2024-02-20T13:02:02.073 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:02.072+0000 7f0576164700 1 -- 172.21.15.71:0/677128477 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05700f6590 msgr2=0x7f05701067c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T13:02:02.073 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:02.072+0000 7f0576164700 1 --2- 172.21.15.71:0/677128477 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05700f6590 0x7f05701067c0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 rx=0x7f0560004170 tx=0x7f0560009ad0).stop 2024-02-20T13:02:02.073 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:02.072+0000 7f0576164700 1 -- 172.21.15.71:0/677128477 shutdown_connections 2024-02-20T13:02:02.074 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:02.072+0000 7f0576164700 1 --2- 172.21.15.71:0/677128477 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f0558031090 0x7f0558033550 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T13:02:02.074 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:02.072+0000 7f0576164700 1 --2- 172.21.15.71:0/677128477 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05700f6590 0x7f05701067c0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T13:02:02.074 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:02.072+0000 7f0576164700 1 -- 172.21.15.71:0/677128477 >> 172.21.15.71:0/677128477 conn(0x7f05700f1cd0 msgr2=0x7f05700f27c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T13:02:02.074 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:02.072+0000 7f0576164700 1 -- 172.21.15.71:0/677128477 shutdown_connections 2024-02-20T13:02:02.074 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:02.073+0000 7f0576164700 1 -- 172.21.15.71:0/677128477 wait complete. 2024-02-20T13:02:02.074 INFO:teuthology.orchestra.run.smithi071.stderr:dumped monmap epoch 1 2024-02-20T13:02:02.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 13:02:02 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T13:02:02.071473+0000 mon.smithi064 (mon.0) 320 : audit [DBG] from='client.? 172.21.15.71:0/677128477' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-20T13:02:03.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 13:02:03 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T13:02:02.103185+0000 mgr.smithi064.vriyor (mgr.14156) 351 : cluster [DBG] pgmap v329: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T13:02:03.752 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-20T13:02:03.753 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/daemon-base:latest-pacific shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 680ee126-cfee-11ee-95bb-87774f69a715 -- ceph mon dump -f json 2024-02-20T13:02:05.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 13:02:05 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T13:02:04.103568+0000 mgr.smithi064.vriyor (mgr.14156) 352 : cluster [DBG] pgmap v330: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T13:02:05.549 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:05.518+0000 7f4bb4e88700 1 -- 172.21.15.71:0/3932092334 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4bb00f5480 msgr2=0x7f4bb00f58a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T13:02:05.550 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:05.518+0000 7f4bb4e88700 1 --2- 172.21.15.71:0/3932092334 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4bb00f5480 0x7f4bb00f58a0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 rx=0x7f4ba0009fb0 tx=0x7f4ba0009b30).stop 2024-02-20T13:02:05.550 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:05.518+0000 7f4bb4e88700 1 -- 172.21.15.71:0/3932092334 shutdown_connections 2024-02-20T13:02:05.550 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:05.518+0000 7f4bb4e88700 1 --2- 172.21.15.71:0/3932092334 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4bb00f5480 0x7f4bb00f58a0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T13:02:05.550 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:05.518+0000 7f4bb4e88700 1 -- 172.21.15.71:0/3932092334 >> 172.21.15.71:0/3932092334 conn(0x7f4bb00f0be0 msgr2=0x7f4bb00f3000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T13:02:05.550 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:05.518+0000 7f4bb4e88700 1 -- 172.21.15.71:0/3932092334 shutdown_connections 2024-02-20T13:02:05.550 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:05.518+0000 7f4bb4e88700 1 -- 172.21.15.71:0/3932092334 wait complete. 2024-02-20T13:02:05.550 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:05.519+0000 7f4bb4e88700 1 Processor -- start 2024-02-20T13:02:05.550 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:05.519+0000 7f4bb4e88700 1 -- start start 2024-02-20T13:02:05.550 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:05.519+0000 7f4bb4e88700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4bb00f5480 0x7f4bb01094f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T13:02:05.550 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:05.519+0000 7f4bb4e88700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bb0060320 con 0x7f4bb00f5480 2024-02-20T13:02:05.550 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:05.519+0000 7f4bae59c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4bb00f5480 0x7f4bb01094f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T13:02:05.550 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:05.519+0000 7f4bae59c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4bb00f5480 0x7f4bb01094f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.71:36512/0 (socket says 172.21.15.71:36512) 2024-02-20T13:02:05.550 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:05.519+0000 7f4bae59c700 1 -- 172.21.15.71:0/3481138996 learned_addr learned my addr 172.21.15.71:0/3481138996 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-02-20T13:02:05.551 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:05.519+0000 7f4bae59c700 1 -- 172.21.15.71:0/3481138996 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ba0005040 con 0x7f4bb00f5480 2024-02-20T13:02:05.551 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:05.519+0000 7f4bae59c700 1 --2- 172.21.15.71:0/3481138996 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4bb00f5480 0x7f4bb01094f0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 rx=0x7f4ba0000c00 tx=0x7f4ba000b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T13:02:05.551 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:05.520+0000 7f4b9effd700 1 -- 172.21.15.71:0/3481138996 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4ba00095f0 con 0x7f4bb00f5480 2024-02-20T13:02:05.551 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:05.520+0000 7f4b9effd700 1 -- 172.21.15.71:0/3481138996 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(25 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f4ba0007e50 con 0x7f4bb00f5480 2024-02-20T13:02:05.551 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:05.520+0000 7f4bb4e88700 1 -- 172.21.15.71:0/3481138996 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4bb0060520 con 0x7f4bb00f5480 2024-02-20T13:02:05.551 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:05.520+0000 7f4b9effd700 1 -- 172.21.15.71:0/3481138996 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4ba0014e70 con 0x7f4bb00f5480 2024-02-20T13:02:05.551 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:05.520+0000 7f4bb4e88700 1 -- 172.21.15.71:0/3481138996 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4bb0109ba0 con 0x7f4bb00f5480 2024-02-20T13:02:05.551 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:05.520+0000 7f4b9effd700 1 -- 172.21.15.71:0/3481138996 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 37151+0+0 (secure 0 0 0) 0x7f4ba00146b0 con 0x7f4bb00f5480 2024-02-20T13:02:05.551 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:05.520+0000 7f4bb4e88700 1 -- 172.21.15.71:0/3481138996 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4bb0045bb0 con 0x7f4bb00f5480 2024-02-20T13:02:05.551 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:05.521+0000 7f4b9effd700 1 --2- 172.21.15.71:0/3481138996 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f4b98031090 0x7f4b98033550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T13:02:05.551 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:05.521+0000 7f4b9effd700 1 -- 172.21.15.71:0/3481138996 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1065+0+0 (secure 0 0 0) 0x7f4ba0041c50 con 0x7f4bb00f5480 2024-02-20T13:02:05.551 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:05.521+0000 7f4badd9b700 1 --2- 172.21.15.71:0/3481138996 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f4b98031090 0x7f4b98033550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T13:02:05.551 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:05.521+0000 7f4badd9b700 1 --2- 172.21.15.71:0/3481138996 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f4b98031090 0x7f4b98033550 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 rx=0x7f4ba40096a0 tx=0x7f4ba4006b40).ready entity=mgr.14156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T13:02:05.551 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:05.526+0000 7f4b9effd700 1 -- 172.21.15.71:0/3481138996 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+153237 (secure 0 0 0) 0x7f4ba001b4a0 con 0x7f4bb00f5480 2024-02-20T13:02:05.734 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:05.732+0000 7f4bb4e88700 1 -- 172.21.15.71:0/3481138996 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f4bb00faf70 con 0x7f4bb00f5480 2024-02-20T13:02:05.734 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:05.733+0000 7f4b9effd700 1 -- 172.21.15.71:0/3481138996 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7f4ba001b2c0 con 0x7f4bb00f5480 2024-02-20T13:02:05.735 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-20T13:02:05.735 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":1,"fsid":"680ee126-cfee-11ee-95bb-87774f69a715","modified":"2024-02-20T12:49:44.699059Z","created":"2024-02-20T12:49:44.699059Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi064","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:3300","nonce":0},{"type":"v1","addr":"172.21.15.64:6789","nonce":0}]},"addr":"172.21.15.64:6789/0","public_addr":"172.21.15.64:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-20T13:02:05.736 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:05.734+0000 7f4bb4e88700 1 -- 172.21.15.71:0/3481138996 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f4b98031090 msgr2=0x7f4b98033550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T13:02:05.736 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:05.734+0000 7f4bb4e88700 1 --2- 172.21.15.71:0/3481138996 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f4b98031090 0x7f4b98033550 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 rx=0x7f4ba40096a0 tx=0x7f4ba4006b40).stop 2024-02-20T13:02:05.736 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:05.734+0000 7f4bb4e88700 1 -- 172.21.15.71:0/3481138996 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4bb00f5480 msgr2=0x7f4bb01094f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T13:02:05.736 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:05.734+0000 7f4bb4e88700 1 --2- 172.21.15.71:0/3481138996 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4bb00f5480 0x7f4bb01094f0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 rx=0x7f4ba0000c00 tx=0x7f4ba000b040).stop 2024-02-20T13:02:05.736 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:05.734+0000 7f4bb4e88700 1 -- 172.21.15.71:0/3481138996 shutdown_connections 2024-02-20T13:02:05.736 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:05.734+0000 7f4bb4e88700 1 --2- 172.21.15.71:0/3481138996 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f4b98031090 0x7f4b98033550 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T13:02:05.737 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:05.734+0000 7f4bb4e88700 1 --2- 172.21.15.71:0/3481138996 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4bb00f5480 0x7f4bb01094f0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T13:02:05.737 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:05.734+0000 7f4bb4e88700 1 -- 172.21.15.71:0/3481138996 >> 172.21.15.71:0/3481138996 conn(0x7f4bb00f0be0 msgr2=0x7f4bb00f16d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T13:02:05.737 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:05.734+0000 7f4bb4e88700 1 -- 172.21.15.71:0/3481138996 shutdown_connections 2024-02-20T13:02:05.737 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:05.734+0000 7f4bb4e88700 1 -- 172.21.15.71:0/3481138996 wait complete. 2024-02-20T13:02:05.737 INFO:teuthology.orchestra.run.smithi071.stderr:dumped monmap epoch 1 2024-02-20T13:02:06.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 13:02:06 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T13:02:05.734565+0000 mon.smithi064 (mon.0) 321 : audit [DBG] from='client.? 172.21.15.71:0/3481138996' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-20T13:02:07.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 13:02:07 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T13:02:06.103991+0000 mgr.smithi064.vriyor (mgr.14156) 353 : cluster [DBG] pgmap v331: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T13:02:07.521 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-20T13:02:07.521 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/daemon-base:latest-pacific shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 680ee126-cfee-11ee-95bb-87774f69a715 -- ceph mon dump -f json 2024-02-20T13:02:09.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 13:02:09 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T13:02:08.104289+0000 mgr.smithi064.vriyor (mgr.14156) 354 : cluster [DBG] pgmap v332: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T13:02:09.636 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:09.625+0000 7efc088cb700 1 -- 172.21.15.71:0/27921328 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efc040f5480 msgr2=0x7efc040f58a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T13:02:09.636 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:09.625+0000 7efc088cb700 1 --2- 172.21.15.71:0/27921328 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efc040f5480 0x7efc040f58a0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 rx=0x7efbec009fb0 tx=0x7efbec009b30).stop 2024-02-20T13:02:09.636 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:09.625+0000 7efc088cb700 1 -- 172.21.15.71:0/27921328 shutdown_connections 2024-02-20T13:02:09.636 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:09.625+0000 7efc088cb700 1 --2- 172.21.15.71:0/27921328 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efc040f5480 0x7efc040f58a0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T13:02:09.636 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:09.625+0000 7efc088cb700 1 -- 172.21.15.71:0/27921328 >> 172.21.15.71:0/27921328 conn(0x7efc040f0be0 msgr2=0x7efc040f3000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T13:02:09.636 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:09.625+0000 7efc088cb700 1 -- 172.21.15.71:0/27921328 shutdown_connections 2024-02-20T13:02:09.636 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:09.625+0000 7efc088cb700 1 -- 172.21.15.71:0/27921328 wait complete. 2024-02-20T13:02:09.637 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:09.626+0000 7efc088cb700 1 Processor -- start 2024-02-20T13:02:09.637 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:09.626+0000 7efc088cb700 1 -- start start 2024-02-20T13:02:09.637 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:09.626+0000 7efc088cb700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efc040f5480 0x7efc041067c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T13:02:09.637 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:09.626+0000 7efc088cb700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc04106cb0 con 0x7efc040f5480 2024-02-20T13:02:09.637 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:09.626+0000 7efc0259c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efc040f5480 0x7efc041067c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T13:02:09.637 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:09.626+0000 7efc0259c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efc040f5480 0x7efc041067c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.71:42020/0 (socket says 172.21.15.71:42020) 2024-02-20T13:02:09.637 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:09.626+0000 7efc0259c700 1 -- 172.21.15.71:0/561624088 learned_addr learned my addr 172.21.15.71:0/561624088 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-02-20T13:02:09.637 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:09.627+0000 7efc0259c700 1 -- 172.21.15.71:0/561624088 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efbec005040 con 0x7efc040f5480 2024-02-20T13:02:09.637 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:09.627+0000 7efc0259c700 1 --2- 172.21.15.71:0/561624088 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efc040f5480 0x7efc041067c0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 rx=0x7efbec0089f0 tx=0x7efbec00b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T13:02:09.637 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:09.627+0000 7efbfaffd700 1 -- 172.21.15.71:0/561624088 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7efbec009320 con 0x7efc040f5480 2024-02-20T13:02:09.637 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:09.627+0000 7efbfaffd700 1 -- 172.21.15.71:0/561624088 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(25 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7efbec013ec0 con 0x7efc040f5480 2024-02-20T13:02:09.637 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:09.627+0000 7efc088cb700 1 -- 172.21.15.71:0/561624088 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc04106eb0 con 0x7efc040f5480 2024-02-20T13:02:09.637 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:09.627+0000 7efbfaffd700 1 -- 172.21.15.71:0/561624088 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7efbec014d30 con 0x7efc040f5480 2024-02-20T13:02:09.637 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:09.627+0000 7efc088cb700 1 -- 172.21.15.71:0/561624088 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc04107340 con 0x7efc040f5480 2024-02-20T13:02:09.637 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:09.628+0000 7efbfaffd700 1 -- 172.21.15.71:0/561624088 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 37151+0+0 (secure 0 0 0) 0x7efbec0095f0 con 0x7efc040f5480 2024-02-20T13:02:09.638 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:09.628+0000 7efc088cb700 1 -- 172.21.15.71:0/561624088 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc04045bb0 con 0x7efc040f5480 2024-02-20T13:02:09.638 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:09.628+0000 7efbfaffd700 1 --2- 172.21.15.71:0/561624088 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7efbf0031090 0x7efbf0033550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T13:02:09.638 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:09.628+0000 7efbfaffd700 1 -- 172.21.15.71:0/561624088 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1065+0+0 (secure 0 0 0) 0x7efbec0424a0 con 0x7efc040f5480 2024-02-20T13:02:09.638 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:09.628+0000 7efc01d9b700 1 --2- 172.21.15.71:0/561624088 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7efbf0031090 0x7efbf0033550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T13:02:09.638 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:09.629+0000 7efc01d9b700 1 --2- 172.21.15.71:0/561624088 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7efbf0031090 0x7efbf0033550 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 rx=0x7efbf40096a0 tx=0x7efbf4006b40).ready entity=mgr.14156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T13:02:09.638 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:09.633+0000 7efbfaffd700 1 -- 172.21.15.71:0/561624088 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+153237 (secure 0 0 0) 0x7efbec01d7f0 con 0x7efc040f5480 2024-02-20T13:02:09.846 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:09.845+0000 7efc088cb700 1 -- 172.21.15.71:0/561624088 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7efc041081f0 con 0x7efc040f5480 2024-02-20T13:02:09.847 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:09.846+0000 7efbfaffd700 1 -- 172.21.15.71:0/561624088 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7efbec01d7f0 con 0x7efc040f5480 2024-02-20T13:02:09.847 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-20T13:02:09.847 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":1,"fsid":"680ee126-cfee-11ee-95bb-87774f69a715","modified":"2024-02-20T12:49:44.699059Z","created":"2024-02-20T12:49:44.699059Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi064","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:3300","nonce":0},{"type":"v1","addr":"172.21.15.64:6789","nonce":0}]},"addr":"172.21.15.64:6789/0","public_addr":"172.21.15.64:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-20T13:02:09.849 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:09.847+0000 7efc088cb700 1 -- 172.21.15.71:0/561624088 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7efbf0031090 msgr2=0x7efbf0033550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T13:02:09.849 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:09.847+0000 7efc088cb700 1 --2- 172.21.15.71:0/561624088 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7efbf0031090 0x7efbf0033550 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 rx=0x7efbf40096a0 tx=0x7efbf4006b40).stop 2024-02-20T13:02:09.849 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:09.847+0000 7efc088cb700 1 -- 172.21.15.71:0/561624088 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efc040f5480 msgr2=0x7efc041067c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T13:02:09.849 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:09.847+0000 7efc088cb700 1 --2- 172.21.15.71:0/561624088 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efc040f5480 0x7efc041067c0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 rx=0x7efbec0089f0 tx=0x7efbec00b040).stop 2024-02-20T13:02:09.849 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:09.847+0000 7efc088cb700 1 -- 172.21.15.71:0/561624088 shutdown_connections 2024-02-20T13:02:09.849 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:09.847+0000 7efc088cb700 1 --2- 172.21.15.71:0/561624088 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7efbf0031090 0x7efbf0033550 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T13:02:09.849 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:09.847+0000 7efc088cb700 1 --2- 172.21.15.71:0/561624088 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efc040f5480 0x7efc041067c0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T13:02:09.849 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:09.847+0000 7efc088cb700 1 -- 172.21.15.71:0/561624088 >> 172.21.15.71:0/561624088 conn(0x7efc040f0be0 msgr2=0x7efc0405fee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T13:02:09.850 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:09.847+0000 7efc088cb700 1 -- 172.21.15.71:0/561624088 shutdown_connections 2024-02-20T13:02:09.850 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:09.847+0000 7efc088cb700 1 -- 172.21.15.71:0/561624088 wait complete. 2024-02-20T13:02:09.850 INFO:teuthology.orchestra.run.smithi071.stderr:dumped monmap epoch 1 2024-02-20T13:02:10.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 13:02:10 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T13:02:09.847564+0000 mon.smithi064 (mon.0) 322 : audit [DBG] from='client.? 172.21.15.71:0/561624088' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-20T13:02:11.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 13:02:11 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T13:02:10.104652+0000 mgr.smithi064.vriyor (mgr.14156) 355 : cluster [DBG] pgmap v333: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T13:02:11.644 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-20T13:02:11.645 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/daemon-base:latest-pacific shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 680ee126-cfee-11ee-95bb-87774f69a715 -- ceph mon dump -f json 2024-02-20T13:02:13.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 13:02:13 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T13:02:12.105047+0000 mgr.smithi064.vriyor (mgr.14156) 356 : cluster [DBG] pgmap v334: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T13:02:13.655 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:13.648+0000 7fb962b50700 1 -- 172.21.15.71:0/2946450501 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb95c0f6590 msgr2=0x7fb95c0f69b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T13:02:13.655 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:13.648+0000 7fb962b50700 1 --2- 172.21.15.71:0/2946450501 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb95c0f6590 0x7fb95c0f69b0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 rx=0x7fb950004760 tx=0x7fb950009ad0).stop 2024-02-20T13:02:13.655 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:13.648+0000 7fb962b50700 1 -- 172.21.15.71:0/2946450501 shutdown_connections 2024-02-20T13:02:13.655 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:13.648+0000 7fb962b50700 1 --2- 172.21.15.71:0/2946450501 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb95c0f6590 0x7fb95c0f69b0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T13:02:13.655 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:13.648+0000 7fb962b50700 1 -- 172.21.15.71:0/2946450501 >> 172.21.15.71:0/2946450501 conn(0x7fb95c0f1cd0 msgr2=0x7fb95c0f4110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T13:02:13.655 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:13.648+0000 7fb962b50700 1 -- 172.21.15.71:0/2946450501 shutdown_connections 2024-02-20T13:02:13.656 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:13.648+0000 7fb962b50700 1 -- 172.21.15.71:0/2946450501 wait complete. 2024-02-20T13:02:13.656 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:13.649+0000 7fb962b50700 1 Processor -- start 2024-02-20T13:02:13.656 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:13.649+0000 7fb962b50700 1 -- start start 2024-02-20T13:02:13.656 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:13.649+0000 7fb962b50700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb95c0f6590 0x7fb95c1067c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T13:02:13.656 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:13.649+0000 7fb962b50700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb95c106cb0 con 0x7fb95c0f6590 2024-02-20T13:02:13.656 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:13.650+0000 7fb9608ec700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb95c0f6590 0x7fb95c1067c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T13:02:13.656 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:13.650+0000 7fb9608ec700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb95c0f6590 0x7fb95c1067c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.71:42032/0 (socket says 172.21.15.71:42032) 2024-02-20T13:02:13.656 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:13.650+0000 7fb9608ec700 1 -- 172.21.15.71:0/1874766341 learned_addr learned my addr 172.21.15.71:0/1874766341 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-02-20T13:02:13.656 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:13.650+0000 7fb9608ec700 1 -- 172.21.15.71:0/1874766341 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb950009d70 con 0x7fb95c0f6590 2024-02-20T13:02:13.656 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:13.650+0000 7fb9608ec700 1 --2- 172.21.15.71:0/1874766341 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb95c0f6590 0x7fb95c1067c0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 rx=0x7fb950004170 tx=0x7fb950009ad0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T13:02:13.656 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:13.650+0000 7fb9597fa700 1 -- 172.21.15.71:0/1874766341 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb95000e040 con 0x7fb95c0f6590 2024-02-20T13:02:13.656 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:13.650+0000 7fb962b50700 1 -- 172.21.15.71:0/1874766341 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb95c106eb0 con 0x7fb95c0f6590 2024-02-20T13:02:13.656 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:13.650+0000 7fb9597fa700 1 -- 172.21.15.71:0/1874766341 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(25 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7fb950008ef0 con 0x7fb95c0f6590 2024-02-20T13:02:13.656 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:13.650+0000 7fb9597fa700 1 -- 172.21.15.71:0/1874766341 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb950014c90 con 0x7fb95c0f6590 2024-02-20T13:02:13.657 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:13.650+0000 7fb962b50700 1 -- 172.21.15.71:0/1874766341 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb95c107340 con 0x7fb95c0f6590 2024-02-20T13:02:13.657 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:13.651+0000 7fb9597fa700 1 -- 172.21.15.71:0/1874766341 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 37151+0+0 (secure 0 0 0) 0x7fb950014420 con 0x7fb95c0f6590 2024-02-20T13:02:13.657 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:13.651+0000 7fb962b50700 1 -- 172.21.15.71:0/1874766341 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb95c045bb0 con 0x7fb95c0f6590 2024-02-20T13:02:13.657 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:13.651+0000 7fb9597fa700 1 --2- 172.21.15.71:0/1874766341 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fb94c031090 0x7fb94c033550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T13:02:13.657 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:13.651+0000 7fb9597fa700 1 -- 172.21.15.71:0/1874766341 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1065+0+0 (secure 0 0 0) 0x7fb950042240 con 0x7fb95c0f6590 2024-02-20T13:02:13.657 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:13.652+0000 7fb95bfff700 1 --2- 172.21.15.71:0/1874766341 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fb94c031090 0x7fb94c033550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T13:02:13.657 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:13.652+0000 7fb95bfff700 1 --2- 172.21.15.71:0/1874766341 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fb94c031090 0x7fb94c033550 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 rx=0x7fb9480096a0 tx=0x7fb948006b40).ready entity=mgr.14156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T13:02:13.657 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:13.656+0000 7fb9597fa700 1 -- 172.21.15.71:0/1874766341 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+153237 (secure 0 0 0) 0x7fb950026300 con 0x7fb95c0f6590 2024-02-20T13:02:13.867 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:13.866+0000 7fb962b50700 1 -- 172.21.15.71:0/1874766341 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fb95c1081f0 con 0x7fb95c0f6590 2024-02-20T13:02:13.868 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:13.866+0000 7fb9597fa700 1 -- 172.21.15.71:0/1874766341 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7fb950012b20 con 0x7fb95c0f6590 2024-02-20T13:02:13.868 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-20T13:02:13.868 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":1,"fsid":"680ee126-cfee-11ee-95bb-87774f69a715","modified":"2024-02-20T12:49:44.699059Z","created":"2024-02-20T12:49:44.699059Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi064","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:3300","nonce":0},{"type":"v1","addr":"172.21.15.64:6789","nonce":0}]},"addr":"172.21.15.64:6789/0","public_addr":"172.21.15.64:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-20T13:02:13.870 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:13.868+0000 7fb962b50700 1 -- 172.21.15.71:0/1874766341 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fb94c031090 msgr2=0x7fb94c033550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T13:02:13.870 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:13.868+0000 7fb962b50700 1 --2- 172.21.15.71:0/1874766341 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fb94c031090 0x7fb94c033550 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 rx=0x7fb9480096a0 tx=0x7fb948006b40).stop 2024-02-20T13:02:13.870 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:13.868+0000 7fb962b50700 1 -- 172.21.15.71:0/1874766341 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb95c0f6590 msgr2=0x7fb95c1067c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T13:02:13.870 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:13.868+0000 7fb962b50700 1 --2- 172.21.15.71:0/1874766341 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb95c0f6590 0x7fb95c1067c0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 rx=0x7fb950004170 tx=0x7fb950009ad0).stop 2024-02-20T13:02:13.870 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:13.868+0000 7fb962b50700 1 -- 172.21.15.71:0/1874766341 shutdown_connections 2024-02-20T13:02:13.870 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:13.868+0000 7fb962b50700 1 --2- 172.21.15.71:0/1874766341 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fb94c031090 0x7fb94c033550 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T13:02:13.871 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:13.868+0000 7fb962b50700 1 --2- 172.21.15.71:0/1874766341 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb95c0f6590 0x7fb95c1067c0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T13:02:13.871 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:13.868+0000 7fb962b50700 1 -- 172.21.15.71:0/1874766341 >> 172.21.15.71:0/1874766341 conn(0x7fb95c0f1cd0 msgr2=0x7fb95c0f27c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T13:02:13.871 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:13.868+0000 7fb962b50700 1 -- 172.21.15.71:0/1874766341 shutdown_connections 2024-02-20T13:02:13.871 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:13.868+0000 7fb962b50700 1 -- 172.21.15.71:0/1874766341 wait complete. 2024-02-20T13:02:13.871 INFO:teuthology.orchestra.run.smithi071.stderr:dumped monmap epoch 1 2024-02-20T13:02:14.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 13:02:14 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T13:02:13.868372+0000 mon.smithi064 (mon.0) 323 : audit [DBG] from='client.? 172.21.15.71:0/1874766341' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-20T13:02:15.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 13:02:15 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T13:02:14.105530+0000 mgr.smithi064.vriyor (mgr.14156) 357 : cluster [DBG] pgmap v335: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T13:02:15.693 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-20T13:02:15.693 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/daemon-base:latest-pacific shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 680ee126-cfee-11ee-95bb-87774f69a715 -- ceph mon dump -f json 2024-02-20T13:02:17.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 13:02:17 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T13:02:16.105857+0000 mgr.smithi064.vriyor (mgr.14156) 358 : cluster [DBG] pgmap v336: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T13:02:17.710 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:17.686+0000 7f4ab5eb1700 1 -- 172.21.15.71:0/1445532147 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4ab0060300 msgr2=0x7f4ab00f91c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T13:02:17.710 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:17.686+0000 7f4ab5eb1700 1 --2- 172.21.15.71:0/1445532147 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4ab0060300 0x7f4ab00f91c0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 rx=0x7f4aa0009fb0 tx=0x7f4aa0009b30).stop 2024-02-20T13:02:17.710 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:17.686+0000 7f4ab5eb1700 1 -- 172.21.15.71:0/1445532147 shutdown_connections 2024-02-20T13:02:17.710 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:17.686+0000 7f4ab5eb1700 1 --2- 172.21.15.71:0/1445532147 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4ab0060300 0x7f4ab00f91c0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T13:02:17.710 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:17.686+0000 7f4ab5eb1700 1 -- 172.21.15.71:0/1445532147 >> 172.21.15.71:0/1445532147 conn(0x7f4ab00f0be0 msgr2=0x7f4ab00f3000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T13:02:17.710 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:17.686+0000 7f4ab5eb1700 1 -- 172.21.15.71:0/1445532147 shutdown_connections 2024-02-20T13:02:17.711 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:17.686+0000 7f4ab5eb1700 1 -- 172.21.15.71:0/1445532147 wait complete. 2024-02-20T13:02:17.711 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:17.686+0000 7f4ab5eb1700 1 Processor -- start 2024-02-20T13:02:17.711 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:17.686+0000 7f4ab5eb1700 1 -- start start 2024-02-20T13:02:17.711 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:17.686+0000 7f4ab5eb1700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4ab0060300 0x7f4ab0106780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T13:02:17.711 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:17.686+0000 7f4ab5eb1700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ab0106c70 con 0x7f4ab0060300 2024-02-20T13:02:17.711 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:17.687+0000 7f4aaf7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4ab0060300 0x7f4ab0106780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T13:02:17.711 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:17.687+0000 7f4aaf7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4ab0060300 0x7f4ab0106780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.71:60292/0 (socket says 172.21.15.71:60292) 2024-02-20T13:02:17.711 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:17.687+0000 7f4aaf7fe700 1 -- 172.21.15.71:0/4126621367 learned_addr learned my addr 172.21.15.71:0/4126621367 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-02-20T13:02:17.712 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:17.687+0000 7f4aaf7fe700 1 -- 172.21.15.71:0/4126621367 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4aa0005040 con 0x7f4ab0060300 2024-02-20T13:02:17.712 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:17.687+0000 7f4aaf7fe700 1 --2- 172.21.15.71:0/4126621367 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4ab0060300 0x7f4ab0106780 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 rx=0x7f4aa0009f80 tx=0x7f4aa000b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T13:02:17.712 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:17.687+0000 7f4aa7fff700 1 -- 172.21.15.71:0/4126621367 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4aa0009320 con 0x7f4ab0060300 2024-02-20T13:02:17.712 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:17.687+0000 7f4ab5eb1700 1 -- 172.21.15.71:0/4126621367 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ab0106e70 con 0x7f4ab0060300 2024-02-20T13:02:17.712 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:17.687+0000 7f4ab5eb1700 1 -- 172.21.15.71:0/4126621367 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ab0107300 con 0x7f4ab0060300 2024-02-20T13:02:17.712 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:17.687+0000 7f4aa7fff700 1 -- 172.21.15.71:0/4126621367 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(25 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f4aa0013680 con 0x7f4ab0060300 2024-02-20T13:02:17.712 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:17.687+0000 7f4aa7fff700 1 -- 172.21.15.71:0/4126621367 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4aa001ed00 con 0x7f4ab0060300 2024-02-20T13:02:17.713 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:17.688+0000 7f4aa7fff700 1 -- 172.21.15.71:0/4126621367 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 37151+0+0 (secure 0 0 0) 0x7f4aa001e420 con 0x7f4ab0060300 2024-02-20T13:02:17.713 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:17.688+0000 7f4ab5eb1700 1 -- 172.21.15.71:0/4126621367 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ab0045bb0 con 0x7f4ab0060300 2024-02-20T13:02:17.713 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:17.688+0000 7f4aa7fff700 1 --2- 172.21.15.71:0/4126621367 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f4a9c031090 0x7f4a9c033550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T13:02:17.713 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:17.688+0000 7f4aa7fff700 1 -- 172.21.15.71:0/4126621367 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1065+0+0 (secure 0 0 0) 0x7f4aa00424a0 con 0x7f4ab0060300 2024-02-20T13:02:17.713 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:17.689+0000 7f4aaeffd700 1 --2- 172.21.15.71:0/4126621367 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f4a9c031090 0x7f4a9c033550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T13:02:17.713 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:17.689+0000 7f4aaeffd700 1 --2- 172.21.15.71:0/4126621367 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f4a9c031090 0x7f4a9c033550 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 rx=0x7f4a980096a0 tx=0x7f4a98006b40).ready entity=mgr.14156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T13:02:17.713 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:17.693+0000 7f4aa7fff700 1 -- 172.21.15.71:0/4126621367 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+153237 (secure 0 0 0) 0x7f4aa00095f0 con 0x7f4ab0060300 2024-02-20T13:02:17.900 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:17.898+0000 7f4ab5eb1700 1 -- 172.21.15.71:0/4126621367 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f4ab01081b0 con 0x7f4ab0060300 2024-02-20T13:02:17.900 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:17.899+0000 7f4aa7fff700 1 -- 172.21.15.71:0/4126621367 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7f4aa002c2d0 con 0x7f4ab0060300 2024-02-20T13:02:17.901 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-20T13:02:17.901 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":1,"fsid":"680ee126-cfee-11ee-95bb-87774f69a715","modified":"2024-02-20T12:49:44.699059Z","created":"2024-02-20T12:49:44.699059Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi064","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:3300","nonce":0},{"type":"v1","addr":"172.21.15.64:6789","nonce":0}]},"addr":"172.21.15.64:6789/0","public_addr":"172.21.15.64:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-20T13:02:17.902 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:17.900+0000 7f4ab5eb1700 1 -- 172.21.15.71:0/4126621367 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f4a9c031090 msgr2=0x7f4a9c033550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T13:02:17.902 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:17.900+0000 7f4ab5eb1700 1 --2- 172.21.15.71:0/4126621367 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f4a9c031090 0x7f4a9c033550 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 rx=0x7f4a980096a0 tx=0x7f4a98006b40).stop 2024-02-20T13:02:17.903 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:17.900+0000 7f4ab5eb1700 1 -- 172.21.15.71:0/4126621367 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4ab0060300 msgr2=0x7f4ab0106780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T13:02:17.903 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:17.900+0000 7f4ab5eb1700 1 --2- 172.21.15.71:0/4126621367 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4ab0060300 0x7f4ab0106780 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 rx=0x7f4aa0009f80 tx=0x7f4aa000b040).stop 2024-02-20T13:02:17.903 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:17.901+0000 7f4ab5eb1700 1 -- 172.21.15.71:0/4126621367 shutdown_connections 2024-02-20T13:02:17.903 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:17.901+0000 7f4ab5eb1700 1 --2- 172.21.15.71:0/4126621367 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f4a9c031090 0x7f4a9c033550 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T13:02:17.903 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:17.901+0000 7f4ab5eb1700 1 --2- 172.21.15.71:0/4126621367 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4ab0060300 0x7f4ab0106780 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T13:02:17.903 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:17.901+0000 7f4ab5eb1700 1 -- 172.21.15.71:0/4126621367 >> 172.21.15.71:0/4126621367 conn(0x7f4ab00f0be0 msgr2=0x7f4ab00f1720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T13:02:17.903 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:17.901+0000 7f4ab5eb1700 1 -- 172.21.15.71:0/4126621367 shutdown_connections 2024-02-20T13:02:17.903 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:17.901+0000 7f4ab5eb1700 1 -- 172.21.15.71:0/4126621367 wait complete. 2024-02-20T13:02:17.903 INFO:teuthology.orchestra.run.smithi071.stderr:dumped monmap epoch 1 2024-02-20T13:02:18.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 13:02:18 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T13:02:17.900903+0000 mon.smithi064 (mon.0) 324 : audit [DBG] from='client.? 172.21.15.71:0/4126621367' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-20T13:02:19.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 13:02:19 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T13:02:18.106282+0000 mgr.smithi064.vriyor (mgr.14156) 359 : cluster [DBG] pgmap v337: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T13:02:19.639 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-20T13:02:19.640 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/daemon-base:latest-pacific shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 680ee126-cfee-11ee-95bb-87774f69a715 -- ceph mon dump -f json 2024-02-20T13:02:21.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 13:02:21 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T13:02:20.106647+0000 mgr.smithi064.vriyor (mgr.14156) 360 : cluster [DBG] pgmap v338: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T13:02:21.600 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:21.597+0000 7f910c0ac700 1 -- 172.21.15.71:0/540619905 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f910406baa0 msgr2=0x7f9104069fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T13:02:21.600 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:21.597+0000 7f910c0ac700 1 --2- 172.21.15.71:0/540619905 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f910406baa0 0x7f9104069fa0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 rx=0x7f90f4004760 tx=0x7f90f4009ad0).stop 2024-02-20T13:02:21.600 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:21.597+0000 7f910c0ac700 1 -- 172.21.15.71:0/540619905 shutdown_connections 2024-02-20T13:02:21.601 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:21.597+0000 7f910c0ac700 1 --2- 172.21.15.71:0/540619905 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f910406baa0 0x7f9104069fa0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T13:02:21.601 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:21.597+0000 7f910c0ac700 1 -- 172.21.15.71:0/540619905 >> 172.21.15.71:0/540619905 conn(0x7f91040f1cd0 msgr2=0x7f91040f4110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T13:02:21.601 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:21.597+0000 7f910c0ac700 1 -- 172.21.15.71:0/540619905 shutdown_connections 2024-02-20T13:02:21.601 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:21.597+0000 7f910c0ac700 1 -- 172.21.15.71:0/540619905 wait complete. 2024-02-20T13:02:21.601 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:21.598+0000 7f910c0ac700 1 Processor -- start 2024-02-20T13:02:21.601 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:21.598+0000 7f910c0ac700 1 -- start start 2024-02-20T13:02:21.601 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:21.598+0000 7f910c0ac700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f910406baa0 0x7f91041089d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T13:02:21.601 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:21.598+0000 7f910c0ac700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9104108ec0 con 0x7f910406baa0 2024-02-20T13:02:21.601 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:21.598+0000 7f9109e48700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f910406baa0 0x7f91041089d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T13:02:21.601 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:21.598+0000 7f9109e48700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f910406baa0 0x7f91041089d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.71:60306/0 (socket says 172.21.15.71:60306) 2024-02-20T13:02:21.601 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:21.598+0000 7f9109e48700 1 -- 172.21.15.71:0/4228269294 learned_addr learned my addr 172.21.15.71:0/4228269294 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-02-20T13:02:21.601 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:21.598+0000 7f9109e48700 1 -- 172.21.15.71:0/4228269294 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90f4009d70 con 0x7f910406baa0 2024-02-20T13:02:21.601 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:21.598+0000 7f9109e48700 1 --2- 172.21.15.71:0/4228269294 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f910406baa0 0x7f91041089d0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 rx=0x7f90f4013f90 tx=0x7f90f4009ad0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T13:02:21.601 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:21.599+0000 7f90fa7fc700 1 -- 172.21.15.71:0/4228269294 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f90f400e040 con 0x7f910406baa0 2024-02-20T13:02:21.602 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:21.599+0000 7f90fa7fc700 1 -- 172.21.15.71:0/4228269294 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(25 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f90f4009680 con 0x7f910406baa0 2024-02-20T13:02:21.602 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:21.599+0000 7f910c0ac700 1 -- 172.21.15.71:0/4228269294 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91041090c0 con 0x7f910406baa0 2024-02-20T13:02:21.602 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:21.599+0000 7f90fa7fc700 1 -- 172.21.15.71:0/4228269294 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f90f4014d20 con 0x7f910406baa0 2024-02-20T13:02:21.602 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:21.599+0000 7f910c0ac700 1 -- 172.21.15.71:0/4228269294 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9104109550 con 0x7f910406baa0 2024-02-20T13:02:21.603 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:21.600+0000 7f90fa7fc700 1 -- 172.21.15.71:0/4228269294 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 37151+0+0 (secure 0 0 0) 0x7f90f401f410 con 0x7f910406baa0 2024-02-20T13:02:21.603 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:21.600+0000 7f90fa7fc700 1 --2- 172.21.15.71:0/4228269294 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f90f0031090 0x7f90f0033550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T13:02:21.603 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:21.600+0000 7f90fa7fc700 1 -- 172.21.15.71:0/4228269294 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1065+0+0 (secure 0 0 0) 0x7f90f40423e0 con 0x7f910406baa0 2024-02-20T13:02:21.603 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:21.600+0000 7f910c0ac700 1 -- 172.21.15.71:0/4228269294 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9104045bb0 con 0x7f910406baa0 2024-02-20T13:02:21.603 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:21.600+0000 7f9109647700 1 --2- 172.21.15.71:0/4228269294 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f90f0031090 0x7f90f0033550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T13:02:21.603 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:21.600+0000 7f9109647700 1 --2- 172.21.15.71:0/4228269294 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f90f0031090 0x7f90f0033550 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 rx=0x7f91000096a0 tx=0x7f9100006b40).ready entity=mgr.14156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T13:02:21.606 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:21.605+0000 7f90fa7fc700 1 -- 172.21.15.71:0/4228269294 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+153237 (secure 0 0 0) 0x7f90f40283f0 con 0x7f910406baa0 2024-02-20T13:02:21.811 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:21.809+0000 7f910c0ac700 1 -- 172.21.15.71:0/4228269294 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f910410a1d0 con 0x7f910406baa0 2024-02-20T13:02:21.812 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:21.810+0000 7f90fa7fc700 1 -- 172.21.15.71:0/4228269294 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7f90f4012330 con 0x7f910406baa0 2024-02-20T13:02:21.812 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-20T13:02:21.812 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":1,"fsid":"680ee126-cfee-11ee-95bb-87774f69a715","modified":"2024-02-20T12:49:44.699059Z","created":"2024-02-20T12:49:44.699059Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi064","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:3300","nonce":0},{"type":"v1","addr":"172.21.15.64:6789","nonce":0}]},"addr":"172.21.15.64:6789/0","public_addr":"172.21.15.64:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-20T13:02:21.813 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:21.812+0000 7f910c0ac700 1 -- 172.21.15.71:0/4228269294 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f90f0031090 msgr2=0x7f90f0033550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T13:02:21.813 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:21.812+0000 7f910c0ac700 1 --2- 172.21.15.71:0/4228269294 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f90f0031090 0x7f90f0033550 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 rx=0x7f91000096a0 tx=0x7f9100006b40).stop 2024-02-20T13:02:21.813 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:21.812+0000 7f910c0ac700 1 -- 172.21.15.71:0/4228269294 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f910406baa0 msgr2=0x7f91041089d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T13:02:21.813 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:21.812+0000 7f910c0ac700 1 --2- 172.21.15.71:0/4228269294 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f910406baa0 0x7f91041089d0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 rx=0x7f90f4013f90 tx=0x7f90f4009ad0).stop 2024-02-20T13:02:21.814 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:21.812+0000 7f910c0ac700 1 -- 172.21.15.71:0/4228269294 shutdown_connections 2024-02-20T13:02:21.814 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:21.812+0000 7f910c0ac700 1 --2- 172.21.15.71:0/4228269294 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f90f0031090 0x7f90f0033550 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T13:02:21.814 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:21.812+0000 7f910c0ac700 1 --2- 172.21.15.71:0/4228269294 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f910406baa0 0x7f91041089d0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T13:02:21.814 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:21.812+0000 7f910c0ac700 1 -- 172.21.15.71:0/4228269294 >> 172.21.15.71:0/4228269294 conn(0x7f91040f1cd0 msgr2=0x7f91040f2810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T13:02:21.814 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:21.812+0000 7f910c0ac700 1 -- 172.21.15.71:0/4228269294 shutdown_connections 2024-02-20T13:02:21.814 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:21.812+0000 7f910c0ac700 1 -- 172.21.15.71:0/4228269294 wait complete. 2024-02-20T13:02:21.814 INFO:teuthology.orchestra.run.smithi071.stderr:dumped monmap epoch 1 2024-02-20T13:02:22.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 13:02:22 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T13:02:21.812101+0000 mon.smithi064 (mon.0) 325 : audit [DBG] from='client.? 172.21.15.71:0/4228269294' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-20T13:02:23.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 13:02:23 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T13:02:22.107052+0000 mgr.smithi064.vriyor (mgr.14156) 361 : cluster [DBG] pgmap v339: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T13:02:23.642 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-20T13:02:23.643 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/daemon-base:latest-pacific shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 680ee126-cfee-11ee-95bb-87774f69a715 -- ceph mon dump -f json 2024-02-20T13:02:25.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 13:02:25 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T13:02:24.107345+0000 mgr.smithi064.vriyor (mgr.14156) 362 : cluster [DBG] pgmap v340: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T13:02:25.667 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:25.665+0000 7fbc49ceb700 1 -- 172.21.15.71:0/497053516 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbc440f6590 msgr2=0x7fbc440f69b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T13:02:25.667 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:25.665+0000 7fbc49ceb700 1 --2- 172.21.15.71:0/497053516 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbc440f6590 0x7fbc440f69b0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 rx=0x7fbc2c004760 tx=0x7fbc2c009ad0).stop 2024-02-20T13:02:25.667 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:25.665+0000 7fbc49ceb700 1 -- 172.21.15.71:0/497053516 shutdown_connections 2024-02-20T13:02:25.667 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:25.665+0000 7fbc49ceb700 1 --2- 172.21.15.71:0/497053516 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbc440f6590 0x7fbc440f69b0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T13:02:25.667 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:25.665+0000 7fbc49ceb700 1 -- 172.21.15.71:0/497053516 >> 172.21.15.71:0/497053516 conn(0x7fbc440f1cd0 msgr2=0x7fbc440f4110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T13:02:25.667 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:25.665+0000 7fbc49ceb700 1 -- 172.21.15.71:0/497053516 shutdown_connections 2024-02-20T13:02:25.668 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:25.665+0000 7fbc49ceb700 1 -- 172.21.15.71:0/497053516 wait complete. 2024-02-20T13:02:25.668 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:25.666+0000 7fbc49ceb700 1 Processor -- start 2024-02-20T13:02:25.668 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:25.666+0000 7fbc49ceb700 1 -- start start 2024-02-20T13:02:25.669 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:25.666+0000 7fbc49ceb700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbc440f6590 0x7fbc441067c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T13:02:25.669 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:25.666+0000 7fbc49ceb700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc44106cb0 con 0x7fbc440f6590 2024-02-20T13:02:25.669 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:25.666+0000 7fbc437fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbc440f6590 0x7fbc441067c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T13:02:25.670 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:25.666+0000 7fbc437fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbc440f6590 0x7fbc441067c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.71:60334/0 (socket says 172.21.15.71:60334) 2024-02-20T13:02:25.670 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:25.666+0000 7fbc437fe700 1 -- 172.21.15.71:0/1322878365 learned_addr learned my addr 172.21.15.71:0/1322878365 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-02-20T13:02:25.670 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:25.666+0000 7fbc437fe700 1 -- 172.21.15.71:0/1322878365 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc2c009d70 con 0x7fbc440f6590 2024-02-20T13:02:25.670 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:25.667+0000 7fbc437fe700 1 --2- 172.21.15.71:0/1322878365 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbc440f6590 0x7fbc441067c0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 rx=0x7fbc2c004170 tx=0x7fbc2c009ad0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T13:02:25.670 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:25.667+0000 7fbc3bfff700 1 -- 172.21.15.71:0/1322878365 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fbc2c00e040 con 0x7fbc440f6590 2024-02-20T13:02:25.670 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:25.667+0000 7fbc49ceb700 1 -- 172.21.15.71:0/1322878365 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc44106eb0 con 0x7fbc440f6590 2024-02-20T13:02:25.670 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:25.667+0000 7fbc3bfff700 1 -- 172.21.15.71:0/1322878365 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(25 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7fbc2c008ef0 con 0x7fbc440f6590 2024-02-20T13:02:25.670 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:25.667+0000 7fbc3bfff700 1 -- 172.21.15.71:0/1322878365 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fbc2c014c90 con 0x7fbc440f6590 2024-02-20T13:02:25.670 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:25.667+0000 7fbc49ceb700 1 -- 172.21.15.71:0/1322878365 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc44107340 con 0x7fbc440f6590 2024-02-20T13:02:25.670 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:25.668+0000 7fbc3bfff700 1 -- 172.21.15.71:0/1322878365 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 37151+0+0 (secure 0 0 0) 0x7fbc2c014420 con 0x7fbc440f6590 2024-02-20T13:02:25.670 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:25.668+0000 7fbc49ceb700 1 -- 172.21.15.71:0/1322878365 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc44045bb0 con 0x7fbc440f6590 2024-02-20T13:02:25.671 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:25.668+0000 7fbc3bfff700 1 --2- 172.21.15.71:0/1322878365 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fbc30031090 0x7fbc30033550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T13:02:25.671 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:25.668+0000 7fbc3bfff700 1 -- 172.21.15.71:0/1322878365 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1065+0+0 (secure 0 0 0) 0x7fbc2c0422e0 con 0x7fbc440f6590 2024-02-20T13:02:25.671 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:25.668+0000 7fbc42ffd700 1 --2- 172.21.15.71:0/1322878365 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fbc30031090 0x7fbc30033550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T13:02:25.671 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:25.668+0000 7fbc42ffd700 1 --2- 172.21.15.71:0/1322878365 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fbc30031090 0x7fbc30033550 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 rx=0x7fbc340096a0 tx=0x7fbc34006b40).ready entity=mgr.14156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T13:02:25.674 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:25.673+0000 7fbc3bfff700 1 -- 172.21.15.71:0/1322878365 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+153237 (secure 0 0 0) 0x7fbc2c012d00 con 0x7fbc440f6590 2024-02-20T13:02:25.880 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:25.879+0000 7fbc49ceb700 1 -- 172.21.15.71:0/1322878365 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fbc441081f0 con 0x7fbc440f6590 2024-02-20T13:02:25.881 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:25.880+0000 7fbc3bfff700 1 -- 172.21.15.71:0/1322878365 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7fbc2c012b20 con 0x7fbc440f6590 2024-02-20T13:02:25.881 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-20T13:02:25.881 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":1,"fsid":"680ee126-cfee-11ee-95bb-87774f69a715","modified":"2024-02-20T12:49:44.699059Z","created":"2024-02-20T12:49:44.699059Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi064","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:3300","nonce":0},{"type":"v1","addr":"172.21.15.64:6789","nonce":0}]},"addr":"172.21.15.64:6789/0","public_addr":"172.21.15.64:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-20T13:02:25.883 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:25.881+0000 7fbc49ceb700 1 -- 172.21.15.71:0/1322878365 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fbc30031090 msgr2=0x7fbc30033550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T13:02:25.883 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:25.881+0000 7fbc49ceb700 1 --2- 172.21.15.71:0/1322878365 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fbc30031090 0x7fbc30033550 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 rx=0x7fbc340096a0 tx=0x7fbc34006b40).stop 2024-02-20T13:02:25.883 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:25.881+0000 7fbc49ceb700 1 -- 172.21.15.71:0/1322878365 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbc440f6590 msgr2=0x7fbc441067c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T13:02:25.883 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:25.881+0000 7fbc49ceb700 1 --2- 172.21.15.71:0/1322878365 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbc440f6590 0x7fbc441067c0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 rx=0x7fbc2c004170 tx=0x7fbc2c009ad0).stop 2024-02-20T13:02:25.883 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:25.881+0000 7fbc49ceb700 1 -- 172.21.15.71:0/1322878365 shutdown_connections 2024-02-20T13:02:25.884 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:25.881+0000 7fbc49ceb700 1 --2- 172.21.15.71:0/1322878365 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fbc30031090 0x7fbc30033550 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T13:02:25.884 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:25.882+0000 7fbc49ceb700 1 --2- 172.21.15.71:0/1322878365 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbc440f6590 0x7fbc441067c0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T13:02:25.884 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:25.882+0000 7fbc49ceb700 1 -- 172.21.15.71:0/1322878365 >> 172.21.15.71:0/1322878365 conn(0x7fbc440f1cd0 msgr2=0x7fbc440f27c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T13:02:25.884 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:25.882+0000 7fbc49ceb700 1 -- 172.21.15.71:0/1322878365 shutdown_connections 2024-02-20T13:02:25.884 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:25.882+0000 7fbc49ceb700 1 -- 172.21.15.71:0/1322878365 wait complete. 2024-02-20T13:02:25.884 INFO:teuthology.orchestra.run.smithi071.stderr:dumped monmap epoch 1 2024-02-20T13:02:26.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 13:02:26 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T13:02:25.881487+0000 mon.smithi064 (mon.0) 326 : audit [DBG] from='client.? 172.21.15.71:0/1322878365' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-20T13:02:27.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 13:02:27 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T13:02:26.107636+0000 mgr.smithi064.vriyor (mgr.14156) 363 : cluster [DBG] pgmap v341: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T13:02:27.687 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-20T13:02:27.687 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/daemon-base:latest-pacific shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 680ee126-cfee-11ee-95bb-87774f69a715 -- ceph mon dump -f json 2024-02-20T13:02:29.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 13:02:29 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T13:02:28.107941+0000 mgr.smithi064.vriyor (mgr.14156) 364 : cluster [DBG] pgmap v342: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T13:02:29.661 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:29.659+0000 7f53d150d700 1 -- 172.21.15.71:0/2583994657 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f53cc0f5480 msgr2=0x7f53cc0f58a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T13:02:29.661 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:29.659+0000 7f53d150d700 1 --2- 172.21.15.71:0/2583994657 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f53cc0f5480 0x7f53cc0f58a0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 rx=0x7f53c0009fb0 tx=0x7f53c0009b30).stop 2024-02-20T13:02:29.661 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:29.659+0000 7f53d150d700 1 -- 172.21.15.71:0/2583994657 shutdown_connections 2024-02-20T13:02:29.661 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:29.659+0000 7f53d150d700 1 --2- 172.21.15.71:0/2583994657 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f53cc0f5480 0x7f53cc0f58a0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T13:02:29.661 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:29.659+0000 7f53d150d700 1 -- 172.21.15.71:0/2583994657 >> 172.21.15.71:0/2583994657 conn(0x7f53cc0f0be0 msgr2=0x7f53cc0f3000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T13:02:29.661 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:29.659+0000 7f53d150d700 1 -- 172.21.15.71:0/2583994657 shutdown_connections 2024-02-20T13:02:29.661 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:29.659+0000 7f53d150d700 1 -- 172.21.15.71:0/2583994657 wait complete. 2024-02-20T13:02:29.661 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:29.660+0000 7f53d150d700 1 Processor -- start 2024-02-20T13:02:29.661 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:29.660+0000 7f53d150d700 1 -- start start 2024-02-20T13:02:29.662 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:29.660+0000 7f53d150d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f53cc0f5480 0x7f53cc1056f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T13:02:29.662 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:29.660+0000 7f53d150d700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53cc105be0 con 0x7f53cc0f5480 2024-02-20T13:02:29.662 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:29.660+0000 7f53caffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f53cc0f5480 0x7f53cc1056f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T13:02:29.662 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:29.660+0000 7f53caffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f53cc0f5480 0x7f53cc1056f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.71:52508/0 (socket says 172.21.15.71:52508) 2024-02-20T13:02:29.662 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:29.661+0000 7f53caffd700 1 -- 172.21.15.71:0/3858332652 learned_addr learned my addr 172.21.15.71:0/3858332652 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-02-20T13:02:29.663 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:29.661+0000 7f53caffd700 1 -- 172.21.15.71:0/3858332652 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53c0005040 con 0x7f53cc0f5480 2024-02-20T13:02:29.664 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:29.661+0000 7f53caffd700 1 --2- 172.21.15.71:0/3858332652 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f53cc0f5480 0x7f53cc1056f0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 rx=0x7f53c00089f0 tx=0x7f53c000b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T13:02:29.664 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:29.661+0000 7f53b37fe700 1 -- 172.21.15.71:0/3858332652 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f53c0009320 con 0x7f53cc0f5480 2024-02-20T13:02:29.664 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:29.661+0000 7f53d150d700 1 -- 172.21.15.71:0/3858332652 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53cc105de0 con 0x7f53cc0f5480 2024-02-20T13:02:29.664 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:29.661+0000 7f53b37fe700 1 -- 172.21.15.71:0/3858332652 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(25 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f53c0013ec0 con 0x7f53cc0f5480 2024-02-20T13:02:29.664 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:29.661+0000 7f53b37fe700 1 -- 172.21.15.71:0/3858332652 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f53c0014d30 con 0x7f53cc0f5480 2024-02-20T13:02:29.664 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:29.661+0000 7f53d150d700 1 -- 172.21.15.71:0/3858332652 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53cc106270 con 0x7f53cc0f5480 2024-02-20T13:02:29.664 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:29.662+0000 7f53b37fe700 1 -- 172.21.15.71:0/3858332652 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 37151+0+0 (secure 0 0 0) 0x7f53c00095f0 con 0x7f53cc0f5480 2024-02-20T13:02:29.664 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:29.662+0000 7f53b37fe700 1 --2- 172.21.15.71:0/3858332652 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f53b4031090 0x7f53b4033550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T13:02:29.667 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:29.662+0000 7f53d150d700 1 -- 172.21.15.71:0/3858332652 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53cc045bb0 con 0x7f53cc0f5480 2024-02-20T13:02:29.667 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:29.662+0000 7f53b37fe700 1 -- 172.21.15.71:0/3858332652 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1065+0+0 (secure 0 0 0) 0x7f53c00423e0 con 0x7f53cc0f5480 2024-02-20T13:02:29.667 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:29.662+0000 7f53ca7fc700 1 --2- 172.21.15.71:0/3858332652 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f53b4031090 0x7f53b4033550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T13:02:29.667 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:29.663+0000 7f53ca7fc700 1 --2- 172.21.15.71:0/3858332652 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f53b4031090 0x7f53b4033550 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 rx=0x7f53bc0096a0 tx=0x7f53bc006b40).ready entity=mgr.14156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T13:02:29.667 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:29.667+0000 7f53b37fe700 1 -- 172.21.15.71:0/3858332652 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+153237 (secure 0 0 0) 0x7f53c001dd70 con 0x7f53cc0f5480 2024-02-20T13:02:29.874 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:29.873+0000 7f53d150d700 1 -- 172.21.15.71:0/3858332652 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f53cc107070 con 0x7f53cc0f5480 2024-02-20T13:02:29.874 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:29.873+0000 7f53b37fe700 1 -- 172.21.15.71:0/3858332652 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7f53c001d380 con 0x7f53cc0f5480 2024-02-20T13:02:29.874 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-20T13:02:29.874 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":1,"fsid":"680ee126-cfee-11ee-95bb-87774f69a715","modified":"2024-02-20T12:49:44.699059Z","created":"2024-02-20T12:49:44.699059Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi064","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:3300","nonce":0},{"type":"v1","addr":"172.21.15.64:6789","nonce":0}]},"addr":"172.21.15.64:6789/0","public_addr":"172.21.15.64:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-20T13:02:29.876 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:29.875+0000 7f53d150d700 1 -- 172.21.15.71:0/3858332652 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f53b4031090 msgr2=0x7f53b4033550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T13:02:29.876 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:29.875+0000 7f53d150d700 1 --2- 172.21.15.71:0/3858332652 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f53b4031090 0x7f53b4033550 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 rx=0x7f53bc0096a0 tx=0x7f53bc006b40).stop 2024-02-20T13:02:29.876 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:29.875+0000 7f53d150d700 1 -- 172.21.15.71:0/3858332652 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f53cc0f5480 msgr2=0x7f53cc1056f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T13:02:29.876 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:29.875+0000 7f53d150d700 1 --2- 172.21.15.71:0/3858332652 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f53cc0f5480 0x7f53cc1056f0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 rx=0x7f53c00089f0 tx=0x7f53c000b040).stop 2024-02-20T13:02:29.876 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:29.875+0000 7f53d150d700 1 -- 172.21.15.71:0/3858332652 shutdown_connections 2024-02-20T13:02:29.876 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:29.875+0000 7f53d150d700 1 --2- 172.21.15.71:0/3858332652 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f53b4031090 0x7f53b4033550 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T13:02:29.876 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:29.875+0000 7f53d150d700 1 --2- 172.21.15.71:0/3858332652 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f53cc0f5480 0x7f53cc1056f0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T13:02:29.876 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:29.875+0000 7f53d150d700 1 -- 172.21.15.71:0/3858332652 >> 172.21.15.71:0/3858332652 conn(0x7f53cc0f0be0 msgr2=0x7f53cc0f16d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T13:02:29.876 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:29.875+0000 7f53d150d700 1 -- 172.21.15.71:0/3858332652 shutdown_connections 2024-02-20T13:02:29.876 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:29.875+0000 7f53d150d700 1 -- 172.21.15.71:0/3858332652 wait complete. 2024-02-20T13:02:29.877 INFO:teuthology.orchestra.run.smithi071.stderr:dumped monmap epoch 1 2024-02-20T13:02:30.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 13:02:30 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T13:02:29.874376+0000 mon.smithi064 (mon.0) 327 : audit [DBG] from='client.? 172.21.15.71:0/3858332652' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-20T13:02:31.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 13:02:31 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T13:02:30.108323+0000 mgr.smithi064.vriyor (mgr.14156) 365 : cluster [DBG] pgmap v343: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T13:02:31.695 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-20T13:02:31.695 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/daemon-base:latest-pacific shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 680ee126-cfee-11ee-95bb-87774f69a715 -- ceph mon dump -f json 2024-02-20T13:02:33.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 13:02:33 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T13:02:32.108671+0000 mgr.smithi064.vriyor (mgr.14156) 366 : cluster [DBG] pgmap v344: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T13:02:33.640 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:33.637+0000 7f6cd3fff700 1 -- 172.21.15.71:0/3237629316 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6cd406baa0 msgr2=0x7f6cd4069fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T13:02:33.640 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:33.637+0000 7f6cd3fff700 1 --2- 172.21.15.71:0/3237629316 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6cd406baa0 0x7f6cd4069fa0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 rx=0x7f6cc4009fb0 tx=0x7f6cc4009b30).stop 2024-02-20T13:02:33.640 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:33.637+0000 7f6cd3fff700 1 -- 172.21.15.71:0/3237629316 shutdown_connections 2024-02-20T13:02:33.640 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:33.637+0000 7f6cd3fff700 1 --2- 172.21.15.71:0/3237629316 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6cd406baa0 0x7f6cd4069fa0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T13:02:33.640 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:33.637+0000 7f6cd3fff700 1 -- 172.21.15.71:0/3237629316 >> 172.21.15.71:0/3237629316 conn(0x7f6cd40f1b10 msgr2=0x7f6cd40f3f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T13:02:33.640 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:33.638+0000 7f6cd3fff700 1 -- 172.21.15.71:0/3237629316 shutdown_connections 2024-02-20T13:02:33.640 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:33.638+0000 7f6cd3fff700 1 -- 172.21.15.71:0/3237629316 wait complete. 2024-02-20T13:02:33.640 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:33.638+0000 7f6cd3fff700 1 Processor -- start 2024-02-20T13:02:33.640 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:33.638+0000 7f6cd3fff700 1 -- start start 2024-02-20T13:02:33.640 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:33.638+0000 7f6cd3fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6cd406baa0 0x7f6cd4108800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T13:02:33.641 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:33.638+0000 7f6cd3fff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cd4108cf0 con 0x7f6cd406baa0 2024-02-20T13:02:33.641 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:33.638+0000 7f6cd2ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6cd406baa0 0x7f6cd4108800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T13:02:33.641 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:33.639+0000 7f6cd2ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6cd406baa0 0x7f6cd4108800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.71:52524/0 (socket says 172.21.15.71:52524) 2024-02-20T13:02:33.641 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:33.639+0000 7f6cd2ffd700 1 -- 172.21.15.71:0/279570622 learned_addr learned my addr 172.21.15.71:0/279570622 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-02-20T13:02:33.641 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:33.639+0000 7f6cd2ffd700 1 -- 172.21.15.71:0/279570622 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cc4005040 con 0x7f6cd406baa0 2024-02-20T13:02:33.641 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:33.639+0000 7f6cd2ffd700 1 --2- 172.21.15.71:0/279570622 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6cd406baa0 0x7f6cd4108800 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 rx=0x7f6cc4009f80 tx=0x7f6cc400b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T13:02:33.641 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:33.639+0000 7f6cbb7fe700 1 -- 172.21.15.71:0/279570622 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6cc4009320 con 0x7f6cd406baa0 2024-02-20T13:02:33.641 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:33.639+0000 7f6cd3fff700 1 -- 172.21.15.71:0/279570622 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6cd4108ef0 con 0x7f6cd406baa0 2024-02-20T13:02:33.641 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:33.639+0000 7f6cbb7fe700 1 -- 172.21.15.71:0/279570622 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(25 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f6cc4013680 con 0x7f6cd406baa0 2024-02-20T13:02:33.641 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:33.639+0000 7f6cbb7fe700 1 -- 172.21.15.71:0/279570622 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6cc401eda0 con 0x7f6cd406baa0 2024-02-20T13:02:33.642 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:33.639+0000 7f6cd3fff700 1 -- 172.21.15.71:0/279570622 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6cd4109380 con 0x7f6cd406baa0 2024-02-20T13:02:33.642 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:33.640+0000 7f6cbb7fe700 1 -- 172.21.15.71:0/279570622 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 37151+0+0 (secure 0 0 0) 0x7f6cc4014420 con 0x7f6cd406baa0 2024-02-20T13:02:33.643 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:33.640+0000 7f6cd3fff700 1 -- 172.21.15.71:0/279570622 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6cb4004fa0 con 0x7f6cd406baa0 2024-02-20T13:02:33.643 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:33.640+0000 7f6cbb7fe700 1 --2- 172.21.15.71:0/279570622 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f6cbc031090 0x7f6cbc033550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T13:02:33.643 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:33.640+0000 7f6cbb7fe700 1 -- 172.21.15.71:0/279570622 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1065+0+0 (secure 0 0 0) 0x7f6cc4042300 con 0x7f6cd406baa0 2024-02-20T13:02:33.643 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:33.640+0000 7f6cd27fc700 1 --2- 172.21.15.71:0/279570622 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f6cbc031090 0x7f6cbc033550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T13:02:33.643 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:33.641+0000 7f6cd27fc700 1 --2- 172.21.15.71:0/279570622 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f6cbc031090 0x7f6cbc033550 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 rx=0x7f6cc80096a0 tx=0x7f6cc8006b40).ready entity=mgr.14156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T13:02:33.645 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:33.645+0000 7f6cbb7fe700 1 -- 172.21.15.71:0/279570622 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+153237 (secure 0 0 0) 0x7f6cc401c6e0 con 0x7f6cd406baa0 2024-02-20T13:02:33.852 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:33.851+0000 7f6cd3fff700 1 -- 172.21.15.71:0/279570622 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f6cb4005670 con 0x7f6cd406baa0 2024-02-20T13:02:33.852 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:33.852+0000 7f6cbb7fe700 1 -- 172.21.15.71:0/279570622 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7f6cc401c320 con 0x7f6cd406baa0 2024-02-20T13:02:33.852 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-20T13:02:33.853 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":1,"fsid":"680ee126-cfee-11ee-95bb-87774f69a715","modified":"2024-02-20T12:49:44.699059Z","created":"2024-02-20T12:49:44.699059Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi064","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:3300","nonce":0},{"type":"v1","addr":"172.21.15.64:6789","nonce":0}]},"addr":"172.21.15.64:6789/0","public_addr":"172.21.15.64:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-20T13:02:33.854 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:33.853+0000 7f6cd3fff700 1 -- 172.21.15.71:0/279570622 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f6cbc031090 msgr2=0x7f6cbc033550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T13:02:33.854 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:33.853+0000 7f6cd3fff700 1 --2- 172.21.15.71:0/279570622 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f6cbc031090 0x7f6cbc033550 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 rx=0x7f6cc80096a0 tx=0x7f6cc8006b40).stop 2024-02-20T13:02:33.854 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:33.853+0000 7f6cd3fff700 1 -- 172.21.15.71:0/279570622 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6cd406baa0 msgr2=0x7f6cd4108800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T13:02:33.854 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:33.853+0000 7f6cd3fff700 1 --2- 172.21.15.71:0/279570622 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6cd406baa0 0x7f6cd4108800 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 rx=0x7f6cc4009f80 tx=0x7f6cc400b040).stop 2024-02-20T13:02:33.854 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:33.853+0000 7f6cd3fff700 1 -- 172.21.15.71:0/279570622 shutdown_connections 2024-02-20T13:02:33.854 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:33.853+0000 7f6cd3fff700 1 --2- 172.21.15.71:0/279570622 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f6cbc031090 0x7f6cbc033550 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T13:02:33.854 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:33.853+0000 7f6cd3fff700 1 --2- 172.21.15.71:0/279570622 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6cd406baa0 0x7f6cd4108800 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T13:02:33.854 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:33.853+0000 7f6cd3fff700 1 -- 172.21.15.71:0/279570622 >> 172.21.15.71:0/279570622 conn(0x7f6cd40f1b10 msgr2=0x7f6cd40f2680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T13:02:33.855 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:33.853+0000 7f6cd3fff700 1 -- 172.21.15.71:0/279570622 shutdown_connections 2024-02-20T13:02:33.855 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:33.853+0000 7f6cd3fff700 1 -- 172.21.15.71:0/279570622 wait complete. 2024-02-20T13:02:33.855 INFO:teuthology.orchestra.run.smithi071.stderr:dumped monmap epoch 1 2024-02-20T13:02:34.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 13:02:34 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T13:02:33.852572+0000 mon.smithi064 (mon.0) 328 : audit [DBG] from='client.? 172.21.15.71:0/279570622' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-20T13:02:35.525 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-20T13:02:35.525 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/daemon-base:latest-pacific shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 680ee126-cfee-11ee-95bb-87774f69a715 -- ceph mon dump -f json 2024-02-20T13:02:35.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 13:02:35 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T13:02:34.109026+0000 mgr.smithi064.vriyor (mgr.14156) 367 : cluster [DBG] pgmap v345: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T13:02:37.322 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:37.320+0000 7f49be3fc700 1 -- 172.21.15.71:0/137553308 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f49b80f5480 msgr2=0x7f49b80f58a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T13:02:37.322 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:37.320+0000 7f49be3fc700 1 --2- 172.21.15.71:0/137553308 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f49b80f5480 0x7f49b80f58a0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 rx=0x7f49ac009fb0 tx=0x7f49ac009b30).stop 2024-02-20T13:02:37.322 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:37.320+0000 7f49be3fc700 1 -- 172.21.15.71:0/137553308 shutdown_connections 2024-02-20T13:02:37.322 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:37.320+0000 7f49be3fc700 1 --2- 172.21.15.71:0/137553308 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f49b80f5480 0x7f49b80f58a0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T13:02:37.322 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:37.320+0000 7f49be3fc700 1 -- 172.21.15.71:0/137553308 >> 172.21.15.71:0/137553308 conn(0x7f49b80f0be0 msgr2=0x7f49b80f3000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T13:02:37.322 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:37.320+0000 7f49be3fc700 1 -- 172.21.15.71:0/137553308 shutdown_connections 2024-02-20T13:02:37.323 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:37.320+0000 7f49be3fc700 1 -- 172.21.15.71:0/137553308 wait complete. 2024-02-20T13:02:37.323 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:37.320+0000 7f49be3fc700 1 Processor -- start 2024-02-20T13:02:37.323 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:37.320+0000 7f49be3fc700 1 -- start start 2024-02-20T13:02:37.323 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:37.321+0000 7f49be3fc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f49b80f5480 0x7f49b81094f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T13:02:37.323 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:37.321+0000 7f49be3fc700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49b8060320 con 0x7f49b80f5480 2024-02-20T13:02:37.323 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:37.321+0000 7f49b7fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f49b80f5480 0x7f49b81094f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T13:02:37.323 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:37.321+0000 7f49b7fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f49b80f5480 0x7f49b81094f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.71:37188/0 (socket says 172.21.15.71:37188) 2024-02-20T13:02:37.323 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:37.321+0000 7f49b7fff700 1 -- 172.21.15.71:0/2360346518 learned_addr learned my addr 172.21.15.71:0/2360346518 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-02-20T13:02:37.323 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:37.321+0000 7f49b7fff700 1 -- 172.21.15.71:0/2360346518 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49ac005040 con 0x7f49b80f5480 2024-02-20T13:02:37.323 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:37.321+0000 7f49b7fff700 1 --2- 172.21.15.71:0/2360346518 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f49b80f5480 0x7f49b81094f0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 rx=0x7f49ac0089f0 tx=0x7f49ac00b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T13:02:37.323 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:37.321+0000 7f49b4ff9700 1 -- 172.21.15.71:0/2360346518 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f49ac009320 con 0x7f49b80f5480 2024-02-20T13:02:37.323 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:37.321+0000 7f49be3fc700 1 -- 172.21.15.71:0/2360346518 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49b8060520 con 0x7f49b80f5480 2024-02-20T13:02:37.323 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:37.321+0000 7f49b4ff9700 1 -- 172.21.15.71:0/2360346518 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(25 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f49ac013ec0 con 0x7f49b80f5480 2024-02-20T13:02:37.323 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:37.321+0000 7f49b4ff9700 1 -- 172.21.15.71:0/2360346518 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f49ac014d30 con 0x7f49b80f5480 2024-02-20T13:02:37.324 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:37.322+0000 7f49be3fc700 1 -- 172.21.15.71:0/2360346518 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49b8109ba0 con 0x7f49b80f5480 2024-02-20T13:02:37.324 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:37.322+0000 7f49b4ff9700 1 -- 172.21.15.71:0/2360346518 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 37151+0+0 (secure 0 0 0) 0x7f49ac0095f0 con 0x7f49b80f5480 2024-02-20T13:02:37.325 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:37.323+0000 7f49b4ff9700 1 --2- 172.21.15.71:0/2360346518 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f49a00310e0 0x7f49a00335a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T13:02:37.325 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:37.323+0000 7f49b4ff9700 1 -- 172.21.15.71:0/2360346518 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1065+0+0 (secure 0 0 0) 0x7f49ac0424e0 con 0x7f49b80f5480 2024-02-20T13:02:37.325 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:37.323+0000 7f49b77fe700 1 --2- 172.21.15.71:0/2360346518 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f49a00310e0 0x7f49a00335a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T13:02:37.325 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:37.323+0000 7f49be3fc700 1 -- 172.21.15.71:0/2360346518 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49b80591e0 con 0x7f49b80f5480 2024-02-20T13:02:37.325 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:37.323+0000 7f49b77fe700 1 --2- 172.21.15.71:0/2360346518 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f49a00310e0 0x7f49a00335a0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 rx=0x7f49a80096a0 tx=0x7f49a8006b40).ready entity=mgr.14156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T13:02:37.328 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:37.328+0000 7f49b4ff9700 1 -- 172.21.15.71:0/2360346518 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+153237 (secure 0 0 0) 0x7f49ac01d7f0 con 0x7f49b80f5480 2024-02-20T13:02:37.535 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:37.535+0000 7f49be3fc700 1 -- 172.21.15.71:0/2360346518 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f49b810a270 con 0x7f49b80f5480 2024-02-20T13:02:37.536 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:37.535+0000 7f49b4ff9700 1 -- 172.21.15.71:0/2360346518 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7f49ac02b020 con 0x7f49b80f5480 2024-02-20T13:02:37.536 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-20T13:02:37.536 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":1,"fsid":"680ee126-cfee-11ee-95bb-87774f69a715","modified":"2024-02-20T12:49:44.699059Z","created":"2024-02-20T12:49:44.699059Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi064","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:3300","nonce":0},{"type":"v1","addr":"172.21.15.64:6789","nonce":0}]},"addr":"172.21.15.64:6789/0","public_addr":"172.21.15.64:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-20T13:02:37.538 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:37.537+0000 7f49be3fc700 1 -- 172.21.15.71:0/2360346518 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f49a00310e0 msgr2=0x7f49a00335a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T13:02:37.538 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:37.537+0000 7f49be3fc700 1 --2- 172.21.15.71:0/2360346518 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f49a00310e0 0x7f49a00335a0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 rx=0x7f49a80096a0 tx=0x7f49a8006b40).stop 2024-02-20T13:02:37.538 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:37.537+0000 7f49be3fc700 1 -- 172.21.15.71:0/2360346518 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f49b80f5480 msgr2=0x7f49b81094f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T13:02:37.538 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:37.537+0000 7f49be3fc700 1 --2- 172.21.15.71:0/2360346518 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f49b80f5480 0x7f49b81094f0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 rx=0x7f49ac0089f0 tx=0x7f49ac00b040).stop 2024-02-20T13:02:37.538 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:37.537+0000 7f49be3fc700 1 -- 172.21.15.71:0/2360346518 shutdown_connections 2024-02-20T13:02:37.539 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:37.537+0000 7f49be3fc700 1 --2- 172.21.15.71:0/2360346518 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f49a00310e0 0x7f49a00335a0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T13:02:37.539 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:37.537+0000 7f49be3fc700 1 --2- 172.21.15.71:0/2360346518 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f49b80f5480 0x7f49b81094f0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T13:02:37.539 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:37.537+0000 7f49be3fc700 1 -- 172.21.15.71:0/2360346518 >> 172.21.15.71:0/2360346518 conn(0x7f49b80f0be0 msgr2=0x7f49b80f16d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T13:02:37.539 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:37.537+0000 7f49be3fc700 1 -- 172.21.15.71:0/2360346518 shutdown_connections 2024-02-20T13:02:37.539 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:37.537+0000 7f49be3fc700 1 -- 172.21.15.71:0/2360346518 wait complete. 2024-02-20T13:02:37.539 INFO:teuthology.orchestra.run.smithi071.stderr:dumped monmap epoch 1 2024-02-20T13:02:37.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 13:02:37 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T13:02:36.109490+0000 mgr.smithi064.vriyor (mgr.14156) 368 : cluster [DBG] pgmap v346: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T13:02:38.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 13:02:38 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T13:02:37.536387+0000 mon.smithi064 (mon.0) 329 : audit [DBG] from='client.? 172.21.15.71:0/2360346518' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-20T13:02:39.326 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-20T13:02:39.327 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/daemon-base:latest-pacific shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 680ee126-cfee-11ee-95bb-87774f69a715 -- ceph mon dump -f json 2024-02-20T13:02:39.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 13:02:39 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T13:02:38.109941+0000 mgr.smithi064.vriyor (mgr.14156) 369 : cluster [DBG] pgmap v347: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T13:02:41.307 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:41.305+0000 7fc94ad14700 1 -- 172.21.15.71:0/4080647169 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc9440f5480 msgr2=0x7fc9440f58a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T13:02:41.308 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:41.305+0000 7fc94ad14700 1 --2- 172.21.15.71:0/4080647169 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc9440f5480 0x7fc9440f58a0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 rx=0x7fc938009fb0 tx=0x7fc938009b30).stop 2024-02-20T13:02:41.308 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:41.305+0000 7fc94ad14700 1 -- 172.21.15.71:0/4080647169 shutdown_connections 2024-02-20T13:02:41.308 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:41.305+0000 7fc94ad14700 1 --2- 172.21.15.71:0/4080647169 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc9440f5480 0x7fc9440f58a0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T13:02:41.308 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:41.305+0000 7fc94ad14700 1 -- 172.21.15.71:0/4080647169 >> 172.21.15.71:0/4080647169 conn(0x7fc9440f0be0 msgr2=0x7fc9440f3000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T13:02:41.308 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:41.305+0000 7fc94ad14700 1 -- 172.21.15.71:0/4080647169 shutdown_connections 2024-02-20T13:02:41.308 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:41.305+0000 7fc94ad14700 1 -- 172.21.15.71:0/4080647169 wait complete. 2024-02-20T13:02:41.308 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:41.306+0000 7fc94ad14700 1 Processor -- start 2024-02-20T13:02:41.308 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:41.306+0000 7fc94ad14700 1 -- start start 2024-02-20T13:02:41.308 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:41.306+0000 7fc94ad14700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc9440f5480 0x7fc9441067f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T13:02:41.308 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:41.306+0000 7fc94ad14700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc944106ce0 con 0x7fc9440f5480 2024-02-20T13:02:41.308 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:41.306+0000 7fc948ab0700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc9440f5480 0x7fc9441067f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T13:02:41.308 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:41.306+0000 7fc948ab0700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc9440f5480 0x7fc9441067f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.71:37204/0 (socket says 172.21.15.71:37204) 2024-02-20T13:02:41.308 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:41.306+0000 7fc948ab0700 1 -- 172.21.15.71:0/2878577252 learned_addr learned my addr 172.21.15.71:0/2878577252 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-02-20T13:02:41.308 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:41.307+0000 7fc948ab0700 1 -- 172.21.15.71:0/2878577252 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc938005040 con 0x7fc9440f5480 2024-02-20T13:02:41.309 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:41.307+0000 7fc948ab0700 1 --2- 172.21.15.71:0/2878577252 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc9440f5480 0x7fc9441067f0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 rx=0x7fc9380089f0 tx=0x7fc93800b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T13:02:41.309 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:41.307+0000 7fc9417fa700 1 -- 172.21.15.71:0/2878577252 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc938009320 con 0x7fc9440f5480 2024-02-20T13:02:41.309 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:41.307+0000 7fc94ad14700 1 -- 172.21.15.71:0/2878577252 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc944106ee0 con 0x7fc9440f5480 2024-02-20T13:02:41.309 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:41.307+0000 7fc9417fa700 1 -- 172.21.15.71:0/2878577252 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(25 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7fc938013ec0 con 0x7fc9440f5480 2024-02-20T13:02:41.309 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:41.307+0000 7fc9417fa700 1 -- 172.21.15.71:0/2878577252 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc938014d30 con 0x7fc9440f5480 2024-02-20T13:02:41.309 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:41.307+0000 7fc94ad14700 1 -- 172.21.15.71:0/2878577252 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc944107370 con 0x7fc9440f5480 2024-02-20T13:02:41.310 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:41.308+0000 7fc9417fa700 1 -- 172.21.15.71:0/2878577252 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 37151+0+0 (secure 0 0 0) 0x7fc9380095f0 con 0x7fc9440f5480 2024-02-20T13:02:41.310 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:41.308+0000 7fc94ad14700 1 -- 172.21.15.71:0/2878577252 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc944045bb0 con 0x7fc9440f5480 2024-02-20T13:02:41.310 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:41.308+0000 7fc9417fa700 1 --2- 172.21.15.71:0/2878577252 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fc934031090 0x7fc934033550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T13:02:41.310 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:41.308+0000 7fc9417fa700 1 -- 172.21.15.71:0/2878577252 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1065+0+0 (secure 0 0 0) 0x7fc9380424a0 con 0x7fc9440f5480 2024-02-20T13:02:41.310 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:41.308+0000 7fc943fff700 1 --2- 172.21.15.71:0/2878577252 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fc934031090 0x7fc934033550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T13:02:41.310 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:41.309+0000 7fc943fff700 1 --2- 172.21.15.71:0/2878577252 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fc934031090 0x7fc934033550 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 rx=0x7fc9300096a0 tx=0x7fc930006b40).ready entity=mgr.14156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T13:02:41.313 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:41.313+0000 7fc9417fa700 1 -- 172.21.15.71:0/2878577252 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+153237 (secure 0 0 0) 0x7fc93801dd70 con 0x7fc9440f5480 2024-02-20T13:02:41.519 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:41.519+0000 7fc94ad14700 1 -- 172.21.15.71:0/2878577252 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fc9440faf70 con 0x7fc9440f5480 2024-02-20T13:02:41.520 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:41.519+0000 7fc9417fa700 1 -- 172.21.15.71:0/2878577252 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7fc93801d380 con 0x7fc9440f5480 2024-02-20T13:02:41.520 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-20T13:02:41.520 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":1,"fsid":"680ee126-cfee-11ee-95bb-87774f69a715","modified":"2024-02-20T12:49:44.699059Z","created":"2024-02-20T12:49:44.699059Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi064","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:3300","nonce":0},{"type":"v1","addr":"172.21.15.64:6789","nonce":0}]},"addr":"172.21.15.64:6789/0","public_addr":"172.21.15.64:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-20T13:02:41.521 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:41.521+0000 7fc94ad14700 1 -- 172.21.15.71:0/2878577252 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fc934031090 msgr2=0x7fc934033550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T13:02:41.521 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:41.521+0000 7fc94ad14700 1 --2- 172.21.15.71:0/2878577252 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fc934031090 0x7fc934033550 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 rx=0x7fc9300096a0 tx=0x7fc930006b40).stop 2024-02-20T13:02:41.521 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:41.521+0000 7fc94ad14700 1 -- 172.21.15.71:0/2878577252 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc9440f5480 msgr2=0x7fc9441067f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T13:02:41.522 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:41.521+0000 7fc94ad14700 1 --2- 172.21.15.71:0/2878577252 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc9440f5480 0x7fc9441067f0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 rx=0x7fc9380089f0 tx=0x7fc93800b040).stop 2024-02-20T13:02:41.522 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:41.521+0000 7fc94ad14700 1 -- 172.21.15.71:0/2878577252 shutdown_connections 2024-02-20T13:02:41.522 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:41.521+0000 7fc94ad14700 1 --2- 172.21.15.71:0/2878577252 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fc934031090 0x7fc934033550 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T13:02:41.522 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:41.521+0000 7fc94ad14700 1 --2- 172.21.15.71:0/2878577252 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc9440f5480 0x7fc9441067f0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T13:02:41.522 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:41.521+0000 7fc94ad14700 1 -- 172.21.15.71:0/2878577252 >> 172.21.15.71:0/2878577252 conn(0x7fc9440f0be0 msgr2=0x7fc9440f16d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T13:02:41.522 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:41.521+0000 7fc94ad14700 1 -- 172.21.15.71:0/2878577252 shutdown_connections 2024-02-20T13:02:41.522 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:41.521+0000 7fc94ad14700 1 -- 172.21.15.71:0/2878577252 wait complete. 2024-02-20T13:02:41.522 INFO:teuthology.orchestra.run.smithi071.stderr:dumped monmap epoch 1 2024-02-20T13:02:41.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 13:02:41 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T13:02:40.110418+0000 mgr.smithi064.vriyor (mgr.14156) 370 : cluster [DBG] pgmap v348: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T13:02:42.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 13:02:42 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T13:02:41.520179+0000 mon.smithi064 (mon.0) 330 : audit [DBG] from='client.? 172.21.15.71:0/2878577252' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-20T13:02:43.202 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-20T13:02:43.202 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/daemon-base:latest-pacific shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 680ee126-cfee-11ee-95bb-87774f69a715 -- ceph mon dump -f json 2024-02-20T13:02:43.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 13:02:43 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T13:02:42.110790+0000 mgr.smithi064.vriyor (mgr.14156) 371 : cluster [DBG] pgmap v349: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T13:02:45.313 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 13:02:45 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T13:02:44.111162+0000 mgr.smithi064.vriyor (mgr.14156) 372 : cluster [DBG] pgmap v350: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T13:02:45.318 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:45.317+0000 7ff24dd4d700 1 -- 172.21.15.71:0/2567027380 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff2480f6590 msgr2=0x7ff2480f69b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T13:02:45.318 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:45.317+0000 7ff24dd4d700 1 --2- 172.21.15.71:0/2567027380 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff2480f6590 0x7ff2480f69b0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 rx=0x7ff23c009fb0 tx=0x7ff23c009b30).stop 2024-02-20T13:02:45.318 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:45.317+0000 7ff24dd4d700 1 -- 172.21.15.71:0/2567027380 shutdown_connections 2024-02-20T13:02:45.318 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:45.317+0000 7ff24dd4d700 1 --2- 172.21.15.71:0/2567027380 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff2480f6590 0x7ff2480f69b0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T13:02:45.318 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:45.317+0000 7ff24dd4d700 1 -- 172.21.15.71:0/2567027380 >> 172.21.15.71:0/2567027380 conn(0x7ff2480f1cf0 msgr2=0x7ff2480f4110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T13:02:45.318 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:45.317+0000 7ff24dd4d700 1 -- 172.21.15.71:0/2567027380 shutdown_connections 2024-02-20T13:02:45.318 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:45.317+0000 7ff24dd4d700 1 -- 172.21.15.71:0/2567027380 wait complete. 2024-02-20T13:02:45.318 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:45.317+0000 7ff24dd4d700 1 Processor -- start 2024-02-20T13:02:45.318 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:45.318+0000 7ff24dd4d700 1 -- start start 2024-02-20T13:02:45.320 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:45.318+0000 7ff24dd4d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff2480f6590 0x7ff248106780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T13:02:45.320 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:45.318+0000 7ff24dd4d700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff248106c70 con 0x7ff2480f6590 2024-02-20T13:02:45.320 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:45.318+0000 7ff2477fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff2480f6590 0x7ff248106780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T13:02:45.320 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:45.318+0000 7ff2477fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff2480f6590 0x7ff248106780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.71:37220/0 (socket says 172.21.15.71:37220) 2024-02-20T13:02:45.320 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:45.318+0000 7ff2477fe700 1 -- 172.21.15.71:0/2607672573 learned_addr learned my addr 172.21.15.71:0/2607672573 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-02-20T13:02:45.321 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:45.318+0000 7ff2477fe700 1 -- 172.21.15.71:0/2607672573 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff23c005040 con 0x7ff2480f6590 2024-02-20T13:02:45.321 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:45.318+0000 7ff2477fe700 1 --2- 172.21.15.71:0/2607672573 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff2480f6590 0x7ff248106780 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 rx=0x7ff23c0089f0 tx=0x7ff23c00b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T13:02:45.321 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:45.319+0000 7ff22ffff700 1 -- 172.21.15.71:0/2607672573 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff23c009320 con 0x7ff2480f6590 2024-02-20T13:02:45.321 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:45.319+0000 7ff24dd4d700 1 -- 172.21.15.71:0/2607672573 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff248106e70 con 0x7ff2480f6590 2024-02-20T13:02:45.321 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:45.319+0000 7ff22ffff700 1 -- 172.21.15.71:0/2607672573 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(25 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7ff23c013ec0 con 0x7ff2480f6590 2024-02-20T13:02:45.321 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:45.319+0000 7ff22ffff700 1 -- 172.21.15.71:0/2607672573 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff23c014d30 con 0x7ff2480f6590 2024-02-20T13:02:45.321 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:45.319+0000 7ff24dd4d700 1 -- 172.21.15.71:0/2607672573 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff248107300 con 0x7ff2480f6590 2024-02-20T13:02:45.321 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:45.319+0000 7ff22ffff700 1 -- 172.21.15.71:0/2607672573 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 37151+0+0 (secure 0 0 0) 0x7ff23c0095f0 con 0x7ff2480f6590 2024-02-20T13:02:45.321 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:45.320+0000 7ff24dd4d700 1 -- 172.21.15.71:0/2607672573 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff248045bb0 con 0x7ff2480f6590 2024-02-20T13:02:45.321 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:45.320+0000 7ff22ffff700 1 --2- 172.21.15.71:0/2607672573 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7ff230031090 0x7ff230033550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T13:02:45.321 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:45.320+0000 7ff22ffff700 1 -- 172.21.15.71:0/2607672573 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1065+0+0 (secure 0 0 0) 0x7ff23c0423e0 con 0x7ff2480f6590 2024-02-20T13:02:45.321 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:45.320+0000 7ff246ffd700 1 --2- 172.21.15.71:0/2607672573 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7ff230031090 0x7ff230033550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T13:02:45.322 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:45.320+0000 7ff246ffd700 1 --2- 172.21.15.71:0/2607672573 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7ff230031090 0x7ff230033550 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 rx=0x7ff2380096a0 tx=0x7ff238006b40).ready entity=mgr.14156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T13:02:45.325 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:45.325+0000 7ff22ffff700 1 -- 172.21.15.71:0/2607672573 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+153237 (secure 0 0 0) 0x7ff23c01d980 con 0x7ff2480f6590 2024-02-20T13:02:45.533 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:45.533+0000 7ff24dd4d700 1 -- 172.21.15.71:0/2607672573 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7ff2481081b0 con 0x7ff2480f6590 2024-02-20T13:02:45.534 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:45.533+0000 7ff22ffff700 1 -- 172.21.15.71:0/2607672573 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7ff23c01d380 con 0x7ff2480f6590 2024-02-20T13:02:45.534 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-20T13:02:45.534 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":1,"fsid":"680ee126-cfee-11ee-95bb-87774f69a715","modified":"2024-02-20T12:49:44.699059Z","created":"2024-02-20T12:49:44.699059Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi064","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:3300","nonce":0},{"type":"v1","addr":"172.21.15.64:6789","nonce":0}]},"addr":"172.21.15.64:6789/0","public_addr":"172.21.15.64:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-20T13:02:45.536 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:45.535+0000 7ff24dd4d700 1 -- 172.21.15.71:0/2607672573 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7ff230031090 msgr2=0x7ff230033550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T13:02:45.536 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:45.535+0000 7ff24dd4d700 1 --2- 172.21.15.71:0/2607672573 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7ff230031090 0x7ff230033550 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 rx=0x7ff2380096a0 tx=0x7ff238006b40).stop 2024-02-20T13:02:45.536 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:45.535+0000 7ff24dd4d700 1 -- 172.21.15.71:0/2607672573 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff2480f6590 msgr2=0x7ff248106780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T13:02:45.536 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:45.535+0000 7ff24dd4d700 1 --2- 172.21.15.71:0/2607672573 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff2480f6590 0x7ff248106780 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 rx=0x7ff23c0089f0 tx=0x7ff23c00b040).stop 2024-02-20T13:02:45.536 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:45.535+0000 7ff24dd4d700 1 -- 172.21.15.71:0/2607672573 shutdown_connections 2024-02-20T13:02:45.536 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:45.535+0000 7ff24dd4d700 1 --2- 172.21.15.71:0/2607672573 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7ff230031090 0x7ff230033550 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T13:02:45.536 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:45.535+0000 7ff24dd4d700 1 --2- 172.21.15.71:0/2607672573 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff2480f6590 0x7ff248106780 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T13:02:45.536 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:45.535+0000 7ff24dd4d700 1 -- 172.21.15.71:0/2607672573 >> 172.21.15.71:0/2607672573 conn(0x7ff2480f1cf0 msgr2=0x7ff2480f27e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T13:02:45.537 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:45.535+0000 7ff24dd4d700 1 -- 172.21.15.71:0/2607672573 shutdown_connections 2024-02-20T13:02:45.537 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:45.535+0000 7ff24dd4d700 1 -- 172.21.15.71:0/2607672573 wait complete. 2024-02-20T13:02:45.537 INFO:teuthology.orchestra.run.smithi071.stderr:dumped monmap epoch 1 2024-02-20T13:02:46.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 13:02:46 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T13:02:45.534267+0000 mon.smithi064 (mon.0) 331 : audit [DBG] from='client.? 172.21.15.71:0/2607672573' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-20T13:02:47.233 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-20T13:02:47.233 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/daemon-base:latest-pacific shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 680ee126-cfee-11ee-95bb-87774f69a715 -- ceph mon dump -f json 2024-02-20T13:02:47.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 13:02:47 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T13:02:46.111544+0000 mgr.smithi064.vriyor (mgr.14156) 373 : cluster [DBG] pgmap v351: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T13:02:49.210 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:49.189+0000 7f128f81f700 1 -- 172.21.15.71:0/435860339 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f12880f4850 msgr2=0x7f12880f4c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T13:02:49.210 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:49.189+0000 7f128f81f700 1 --2- 172.21.15.71:0/435860339 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f12880f4850 0x7f12880f4c70 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 rx=0x7f1278009fb0 tx=0x7f1278009b30).stop 2024-02-20T13:02:49.210 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:49.189+0000 7f128f81f700 1 -- 172.21.15.71:0/435860339 shutdown_connections 2024-02-20T13:02:49.210 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:49.189+0000 7f128f81f700 1 --2- 172.21.15.71:0/435860339 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f12880f4850 0x7f12880f4c70 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T13:02:49.210 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:49.189+0000 7f128f81f700 1 -- 172.21.15.71:0/435860339 >> 172.21.15.71:0/435860339 conn(0x7f12880f0390 msgr2=0x7f12880f27b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T13:02:49.210 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:49.189+0000 7f128f81f700 1 -- 172.21.15.71:0/435860339 shutdown_connections 2024-02-20T13:02:49.211 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:49.189+0000 7f128f81f700 1 -- 172.21.15.71:0/435860339 wait complete. 2024-02-20T13:02:49.211 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:49.190+0000 7f128f81f700 1 Processor -- start 2024-02-20T13:02:49.211 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:49.190+0000 7f128f81f700 1 -- start start 2024-02-20T13:02:49.211 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:49.190+0000 7f128f81f700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f12880f4850 0x7f12881067b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T13:02:49.211 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:49.190+0000 7f128f81f700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1288106ca0 con 0x7f12880f4850 2024-02-20T13:02:49.211 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:49.190+0000 7f128d5bb700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f12880f4850 0x7f12881067b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T13:02:49.211 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:49.190+0000 7f128d5bb700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f12880f4850 0x7f12881067b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.71:56138/0 (socket says 172.21.15.71:56138) 2024-02-20T13:02:49.211 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:49.190+0000 7f128d5bb700 1 -- 172.21.15.71:0/2274748294 learned_addr learned my addr 172.21.15.71:0/2274748294 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-02-20T13:02:49.211 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:49.190+0000 7f128d5bb700 1 -- 172.21.15.71:0/2274748294 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1278005040 con 0x7f12880f4850 2024-02-20T13:02:49.211 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:49.190+0000 7f128d5bb700 1 --2- 172.21.15.71:0/2274748294 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f12880f4850 0x7f12881067b0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 rx=0x7f12780089f0 tx=0x7f127800b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T13:02:49.211 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:49.190+0000 7f127dffb700 1 -- 172.21.15.71:0/2274748294 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1278009320 con 0x7f12880f4850 2024-02-20T13:02:49.211 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:49.190+0000 7f128f81f700 1 -- 172.21.15.71:0/2274748294 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1288106ea0 con 0x7f12880f4850 2024-02-20T13:02:49.211 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:49.191+0000 7f127dffb700 1 -- 172.21.15.71:0/2274748294 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(25 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7f1278013ec0 con 0x7f12880f4850 2024-02-20T13:02:49.211 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:49.191+0000 7f127dffb700 1 -- 172.21.15.71:0/2274748294 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1278014d30 con 0x7f12880f4850 2024-02-20T13:02:49.211 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:49.191+0000 7f128f81f700 1 -- 172.21.15.71:0/2274748294 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1288107330 con 0x7f12880f4850 2024-02-20T13:02:49.212 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:49.191+0000 7f127dffb700 1 -- 172.21.15.71:0/2274748294 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 37151+0+0 (secure 0 0 0) 0x7f12780263a0 con 0x7f12880f4850 2024-02-20T13:02:49.212 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:49.192+0000 7f127dffb700 1 --2- 172.21.15.71:0/2274748294 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f12740310e0 0x7f12740335a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T13:02:49.212 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:49.192+0000 7f127dffb700 1 -- 172.21.15.71:0/2274748294 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1065+0+0 (secure 0 0 0) 0x7f1278042460 con 0x7f12880f4850 2024-02-20T13:02:49.212 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:49.192+0000 7f128f81f700 1 -- 172.21.15.71:0/2274748294 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1288045bb0 con 0x7f12880f4850 2024-02-20T13:02:49.212 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:49.192+0000 7f128cdba700 1 --2- 172.21.15.71:0/2274748294 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f12740310e0 0x7f12740335a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T13:02:49.212 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:49.192+0000 7f128cdba700 1 --2- 172.21.15.71:0/2274748294 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f12740310e0 0x7f12740335a0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 rx=0x7f12840096a0 tx=0x7f1284006b40).ready entity=mgr.14156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T13:02:49.212 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:49.197+0000 7f127dffb700 1 -- 172.21.15.71:0/2274748294 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+153237 (secure 0 0 0) 0x7f12780266d0 con 0x7f12880f4850 2024-02-20T13:02:49.406 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:49.405+0000 7f128f81f700 1 -- 172.21.15.71:0/2274748294 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f12880fc170 con 0x7f12880f4850 2024-02-20T13:02:49.406 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:49.406+0000 7f127dffb700 1 -- 172.21.15.71:0/2274748294 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7f127802b020 con 0x7f12880f4850 2024-02-20T13:02:49.407 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-20T13:02:49.407 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":1,"fsid":"680ee126-cfee-11ee-95bb-87774f69a715","modified":"2024-02-20T12:49:44.699059Z","created":"2024-02-20T12:49:44.699059Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi064","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:3300","nonce":0},{"type":"v1","addr":"172.21.15.64:6789","nonce":0}]},"addr":"172.21.15.64:6789/0","public_addr":"172.21.15.64:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-20T13:02:49.408 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:49.408+0000 7f128f81f700 1 -- 172.21.15.71:0/2274748294 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f12740310e0 msgr2=0x7f12740335a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T13:02:49.409 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:49.408+0000 7f128f81f700 1 --2- 172.21.15.71:0/2274748294 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f12740310e0 0x7f12740335a0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 rx=0x7f12840096a0 tx=0x7f1284006b40).stop 2024-02-20T13:02:49.409 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:49.408+0000 7f128f81f700 1 -- 172.21.15.71:0/2274748294 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f12880f4850 msgr2=0x7f12881067b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T13:02:49.409 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:49.408+0000 7f128f81f700 1 --2- 172.21.15.71:0/2274748294 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f12880f4850 0x7f12881067b0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 rx=0x7f12780089f0 tx=0x7f127800b040).stop 2024-02-20T13:02:49.409 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:49.408+0000 7f128f81f700 1 -- 172.21.15.71:0/2274748294 shutdown_connections 2024-02-20T13:02:49.409 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:49.408+0000 7f128f81f700 1 --2- 172.21.15.71:0/2274748294 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7f12740310e0 0x7f12740335a0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T13:02:49.409 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:49.408+0000 7f128f81f700 1 --2- 172.21.15.71:0/2274748294 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f12880f4850 0x7f12881067b0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T13:02:49.409 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:49.408+0000 7f128f81f700 1 -- 172.21.15.71:0/2274748294 >> 172.21.15.71:0/2274748294 conn(0x7f12880f0390 msgr2=0x7f12880f0e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T13:02:49.409 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:49.408+0000 7f128f81f700 1 -- 172.21.15.71:0/2274748294 shutdown_connections 2024-02-20T13:02:49.409 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:49.408+0000 7f128f81f700 1 -- 172.21.15.71:0/2274748294 wait complete. 2024-02-20T13:02:49.410 INFO:teuthology.orchestra.run.smithi071.stderr:dumped monmap epoch 1 2024-02-20T13:02:49.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 13:02:49 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T13:02:48.111945+0000 mgr.smithi064.vriyor (mgr.14156) 374 : cluster [DBG] pgmap v352: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T13:02:50.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 13:02:50 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T13:02:49.407067+0000 mon.smithi064 (mon.0) 332 : audit [DBG] from='client.? 172.21.15.71:0/2274748294' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-20T13:02:51.129 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-20T13:02:51.130 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/daemon-base:latest-pacific shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 680ee126-cfee-11ee-95bb-87774f69a715 -- ceph mon dump -f json 2024-02-20T13:02:51.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 13:02:51 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T13:02:50.112257+0000 mgr.smithi064.vriyor (mgr.14156) 375 : cluster [DBG] pgmap v353: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T13:02:53.126 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:53.125+0000 7fab5df1e700 1 -- 172.21.15.71:0/817323430 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fab580f63e0 msgr2=0x7fab580f6800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T13:02:53.126 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:53.125+0000 7fab5df1e700 1 --2- 172.21.15.71:0/817323430 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fab580f63e0 0x7fab580f6800 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 rx=0x7fab4c009fb0 tx=0x7fab4c009b30).stop 2024-02-20T13:02:53.127 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:53.125+0000 7fab5df1e700 1 -- 172.21.15.71:0/817323430 shutdown_connections 2024-02-20T13:02:53.127 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:53.125+0000 7fab5df1e700 1 --2- 172.21.15.71:0/817323430 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fab580f63e0 0x7fab580f6800 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T13:02:53.127 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:53.125+0000 7fab5df1e700 1 -- 172.21.15.71:0/817323430 >> 172.21.15.71:0/817323430 conn(0x7fab580f1b40 msgr2=0x7fab580f3f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T13:02:53.127 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:53.125+0000 7fab5df1e700 1 -- 172.21.15.71:0/817323430 shutdown_connections 2024-02-20T13:02:53.127 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:53.125+0000 7fab5df1e700 1 -- 172.21.15.71:0/817323430 wait complete. 2024-02-20T13:02:53.127 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:53.126+0000 7fab5df1e700 1 Processor -- start 2024-02-20T13:02:53.127 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:53.126+0000 7fab5df1e700 1 -- start start 2024-02-20T13:02:53.127 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:53.126+0000 7fab5df1e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fab580f63e0 0x7fab58106620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T13:02:53.127 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:53.126+0000 7fab5df1e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab58106b10 con 0x7fab580f63e0 2024-02-20T13:02:53.129 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:53.126+0000 7fab5cf1c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fab580f63e0 0x7fab58106620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T13:02:53.129 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:53.126+0000 7fab5cf1c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fab580f63e0 0x7fab58106620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.71:56148/0 (socket says 172.21.15.71:56148) 2024-02-20T13:02:53.129 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:53.126+0000 7fab5cf1c700 1 -- 172.21.15.71:0/850153815 learned_addr learned my addr 172.21.15.71:0/850153815 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-02-20T13:02:53.129 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:53.127+0000 7fab5cf1c700 1 -- 172.21.15.71:0/850153815 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab4c005040 con 0x7fab580f63e0 2024-02-20T13:02:53.129 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:53.127+0000 7fab5cf1c700 1 --2- 172.21.15.71:0/850153815 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fab580f63e0 0x7fab58106620 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 rx=0x7fab4c0089f0 tx=0x7fab4c00b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T13:02:53.129 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:53.127+0000 7fab557fa700 1 -- 172.21.15.71:0/850153815 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fab4c009320 con 0x7fab580f63e0 2024-02-20T13:02:53.129 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:53.127+0000 7fab5df1e700 1 -- 172.21.15.71:0/850153815 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab58106d10 con 0x7fab580f63e0 2024-02-20T13:02:53.129 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:53.127+0000 7fab5df1e700 1 -- 172.21.15.71:0/850153815 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab581071a0 con 0x7fab580f63e0 2024-02-20T13:02:53.129 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:53.127+0000 7fab557fa700 1 -- 172.21.15.71:0/850153815 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(25 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7fab4c004490 con 0x7fab580f63e0 2024-02-20T13:02:53.130 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:53.127+0000 7fab557fa700 1 -- 172.21.15.71:0/850153815 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fab4c014c50 con 0x7fab580f63e0 2024-02-20T13:02:53.130 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:53.128+0000 7fab557fa700 1 -- 172.21.15.71:0/850153815 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 37151+0+0 (secure 0 0 0) 0x7fab4c014420 con 0x7fab580f63e0 2024-02-20T13:02:53.130 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:53.128+0000 7fab5df1e700 1 -- 172.21.15.71:0/850153815 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab38004fa0 con 0x7fab580f63e0 2024-02-20T13:02:53.130 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:53.128+0000 7fab557fa700 1 --2- 172.21.15.71:0/850153815 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fab48031090 0x7fab48033550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T13:02:53.130 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:53.128+0000 7fab557fa700 1 -- 172.21.15.71:0/850153815 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1065+0+0 (secure 0 0 0) 0x7fab4c042620 con 0x7fab580f63e0 2024-02-20T13:02:53.130 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:53.129+0000 7fab57fff700 1 --2- 172.21.15.71:0/850153815 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fab48031090 0x7fab48033550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T13:02:53.130 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:53.129+0000 7fab57fff700 1 --2- 172.21.15.71:0/850153815 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fab48031090 0x7fab48033550 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 rx=0x7fab440096a0 tx=0x7fab44006b40).ready entity=mgr.14156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T13:02:53.134 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:53.133+0000 7fab557fa700 1 -- 172.21.15.71:0/850153815 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+153237 (secure 0 0 0) 0x7fab4c01c790 con 0x7fab580f63e0 2024-02-20T13:02:53.340 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:53.340+0000 7fab5df1e700 1 -- 172.21.15.71:0/850153815 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fab38005670 con 0x7fab580f63e0 2024-02-20T13:02:53.341 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:53.340+0000 7fab557fa700 1 -- 172.21.15.71:0/850153815 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7fab4c025e40 con 0x7fab580f63e0 2024-02-20T13:02:53.341 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-20T13:02:53.342 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":1,"fsid":"680ee126-cfee-11ee-95bb-87774f69a715","modified":"2024-02-20T12:49:44.699059Z","created":"2024-02-20T12:49:44.699059Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi064","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:3300","nonce":0},{"type":"v1","addr":"172.21.15.64:6789","nonce":0}]},"addr":"172.21.15.64:6789/0","public_addr":"172.21.15.64:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-20T13:02:53.343 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:53.342+0000 7fab5df1e700 1 -- 172.21.15.71:0/850153815 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fab48031090 msgr2=0x7fab48033550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T13:02:53.343 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:53.342+0000 7fab5df1e700 1 --2- 172.21.15.71:0/850153815 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fab48031090 0x7fab48033550 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 rx=0x7fab440096a0 tx=0x7fab44006b40).stop 2024-02-20T13:02:53.344 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:53.342+0000 7fab5df1e700 1 -- 172.21.15.71:0/850153815 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fab580f63e0 msgr2=0x7fab58106620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T13:02:53.344 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:53.342+0000 7fab5df1e700 1 --2- 172.21.15.71:0/850153815 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fab580f63e0 0x7fab58106620 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 rx=0x7fab4c0089f0 tx=0x7fab4c00b040).stop 2024-02-20T13:02:53.344 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:53.342+0000 7fab5df1e700 1 -- 172.21.15.71:0/850153815 shutdown_connections 2024-02-20T13:02:53.344 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:53.342+0000 7fab5df1e700 1 --2- 172.21.15.71:0/850153815 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fab48031090 0x7fab48033550 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T13:02:53.344 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:53.342+0000 7fab5df1e700 1 --2- 172.21.15.71:0/850153815 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fab580f63e0 0x7fab58106620 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T13:02:53.344 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:53.342+0000 7fab5df1e700 1 -- 172.21.15.71:0/850153815 >> 172.21.15.71:0/850153815 conn(0x7fab580f1b40 msgr2=0x7fab580f2660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T13:02:53.344 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:53.342+0000 7fab5df1e700 1 -- 172.21.15.71:0/850153815 shutdown_connections 2024-02-20T13:02:53.344 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:53.342+0000 7fab5df1e700 1 -- 172.21.15.71:0/850153815 wait complete. 2024-02-20T13:02:53.345 INFO:teuthology.orchestra.run.smithi071.stderr:dumped monmap epoch 1 2024-02-20T13:02:53.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 13:02:53 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T13:02:52.112562+0000 mgr.smithi064.vriyor (mgr.14156) 376 : cluster [DBG] pgmap v354: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T13:02:54.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 13:02:54 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T13:02:53.341161+0000 mon.smithi064 (mon.0) 333 : audit [DBG] from='client.? 172.21.15.71:0/850153815' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-20T13:02:54.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 13:02:54 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T13:02:53.855851+0000 mon.smithi064 (mon.0) 334 : audit [INF] from='mgr.14156 172.21.15.64:0/3284974785' entity='mgr.smithi064.vriyor' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi064.vriyor/mirror_snapshot_schedule"}]: dispatch 2024-02-20T13:02:54.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 13:02:54 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T13:02:53.856079+0000 mon.smithi064 (mon.0) 335 : audit [INF] from='mgr.14156 172.21.15.64:0/3284974785' entity='mgr.smithi064.vriyor' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi064.vriyor/trash_purge_schedule"}]: dispatch 2024-02-20T13:02:54.955 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-20T13:02:54.956 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/daemon-base:latest-pacific shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 680ee126-cfee-11ee-95bb-87774f69a715 -- ceph mon dump -f json 2024-02-20T13:02:55.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 13:02:55 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T13:02:54.112933+0000 mgr.smithi064.vriyor (mgr.14156) 377 : cluster [DBG] pgmap v355: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T13:02:56.869 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:56.866+0000 7fa3af9f4700 1 -- 172.21.15.71:0/107967396 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa3a80602c0 msgr2=0x7fa3a80f9200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T13:02:56.869 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:56.866+0000 7fa3af9f4700 1 --2- 172.21.15.71:0/107967396 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa3a80602c0 0x7fa3a80f9200 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 rx=0x7fa398009fb0 tx=0x7fa398009b30).stop 2024-02-20T13:02:56.869 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:56.866+0000 7fa3af9f4700 1 -- 172.21.15.71:0/107967396 shutdown_connections 2024-02-20T13:02:56.869 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:56.866+0000 7fa3af9f4700 1 --2- 172.21.15.71:0/107967396 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa3a80602c0 0x7fa3a80f9200 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T13:02:56.869 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:56.866+0000 7fa3af9f4700 1 -- 172.21.15.71:0/107967396 >> 172.21.15.71:0/107967396 conn(0x7fa3a80f0be0 msgr2=0x7fa3a80f3000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T13:02:56.870 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:56.867+0000 7fa3af9f4700 1 -- 172.21.15.71:0/107967396 shutdown_connections 2024-02-20T13:02:56.870 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:56.867+0000 7fa3af9f4700 1 -- 172.21.15.71:0/107967396 wait complete. 2024-02-20T13:02:56.870 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:56.867+0000 7fa3af9f4700 1 Processor -- start 2024-02-20T13:02:56.870 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:56.867+0000 7fa3af9f4700 1 -- start start 2024-02-20T13:02:56.870 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:56.867+0000 7fa3af9f4700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa3a80602c0 0x7fa3a8106730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T13:02:56.870 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:56.867+0000 7fa3af9f4700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3a8106c20 con 0x7fa3a80602c0 2024-02-20T13:02:56.870 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:56.867+0000 7fa3ad790700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa3a80602c0 0x7fa3a8106730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T13:02:56.870 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:56.867+0000 7fa3ad790700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa3a80602c0 0x7fa3a8106730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.71:56168/0 (socket says 172.21.15.71:56168) 2024-02-20T13:02:56.870 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:56.867+0000 7fa3ad790700 1 -- 172.21.15.71:0/2664363309 learned_addr learned my addr 172.21.15.71:0/2664363309 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-02-20T13:02:56.870 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:56.868+0000 7fa3ad790700 1 -- 172.21.15.71:0/2664363309 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa398005040 con 0x7fa3a80602c0 2024-02-20T13:02:56.870 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:56.868+0000 7fa3ad790700 1 --2- 172.21.15.71:0/2664363309 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa3a80602c0 0x7fa3a8106730 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 rx=0x7fa398013fd0 tx=0x7fa398009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T13:02:56.870 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:56.868+0000 7fa39dffb700 1 -- 172.21.15.71:0/2664363309 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa398018070 con 0x7fa3a80602c0 2024-02-20T13:02:56.870 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:56.868+0000 7fa3af9f4700 1 -- 172.21.15.71:0/2664363309 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3a8106e20 con 0x7fa3a80602c0 2024-02-20T13:02:56.870 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:56.868+0000 7fa39dffb700 1 -- 172.21.15.71:0/2664363309 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(25 keys) v1 ==== 1054+0+0 (secure 0 0 0) 0x7fa398007b10 con 0x7fa3a80602c0 2024-02-20T13:02:56.871 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:56.868+0000 7fa39dffb700 1 -- 172.21.15.71:0/2664363309 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa39801c3c0 con 0x7fa3a80602c0 2024-02-20T13:02:56.871 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:56.868+0000 7fa3af9f4700 1 -- 172.21.15.71:0/2664363309 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3a81072b0 con 0x7fa3a80602c0 2024-02-20T13:02:56.871 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:56.869+0000 7fa39dffb700 1 -- 172.21.15.71:0/2664363309 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 37151+0+0 (secure 0 0 0) 0x7fa39801c640 con 0x7fa3a80602c0 2024-02-20T13:02:56.871 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:56.869+0000 7fa39dffb700 1 --2- 172.21.15.71:0/2664363309 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fa394031090 0x7fa394033550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-20T13:02:56.871 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:56.869+0000 7fa39dffb700 1 -- 172.21.15.71:0/2664363309 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1065+0+0 (secure 0 0 0) 0x7fa398042a00 con 0x7fa3a80602c0 2024-02-20T13:02:56.872 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:56.869+0000 7fa3af9f4700 1 -- 172.21.15.71:0/2664363309 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3a80591e0 con 0x7fa3a80602c0 2024-02-20T13:02:56.872 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:56.869+0000 7fa3acf8f700 1 --2- 172.21.15.71:0/2664363309 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fa394031090 0x7fa394033550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-20T13:02:56.872 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:56.870+0000 7fa3acf8f700 1 --2- 172.21.15.71:0/2664363309 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fa394031090 0x7fa394033550 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 rx=0x7fa3a40096a0 tx=0x7fa3a4006b40).ready entity=mgr.14156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-20T13:02:56.875 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:56.874+0000 7fa39dffb700 1 -- 172.21.15.71:0/2664363309 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+153237 (secure 0 0 0) 0x7fa398024300 con 0x7fa3a80602c0 2024-02-20T13:02:57.082 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:57.081+0000 7fa3af9f4700 1 -- 172.21.15.71:0/2664363309 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fa3a80fc070 con 0x7fa3a80602c0 2024-02-20T13:02:57.082 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:57.082+0000 7fa39dffb700 1 -- 172.21.15.71:0/2664363309 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7fa39801bb60 con 0x7fa3a80602c0 2024-02-20T13:02:57.082 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-20T13:02:57.083 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":1,"fsid":"680ee126-cfee-11ee-95bb-87774f69a715","modified":"2024-02-20T12:49:44.699059Z","created":"2024-02-20T12:49:44.699059Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi064","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:3300","nonce":0},{"type":"v1","addr":"172.21.15.64:6789","nonce":0}]},"addr":"172.21.15.64:6789/0","public_addr":"172.21.15.64:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-20T13:02:57.084 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:57.083+0000 7fa3af9f4700 1 -- 172.21.15.71:0/2664363309 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fa394031090 msgr2=0x7fa394033550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T13:02:57.084 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:57.083+0000 7fa3af9f4700 1 --2- 172.21.15.71:0/2664363309 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fa394031090 0x7fa394033550 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 rx=0x7fa3a40096a0 tx=0x7fa3a4006b40).stop 2024-02-20T13:02:57.084 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:57.083+0000 7fa3af9f4700 1 -- 172.21.15.71:0/2664363309 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa3a80602c0 msgr2=0x7fa3a8106730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-20T13:02:57.084 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:57.083+0000 7fa3af9f4700 1 --2- 172.21.15.71:0/2664363309 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa3a80602c0 0x7fa3a8106730 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 rx=0x7fa398013fd0 tx=0x7fa398009b30).stop 2024-02-20T13:02:57.084 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:57.083+0000 7fa3af9f4700 1 -- 172.21.15.71:0/2664363309 shutdown_connections 2024-02-20T13:02:57.084 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:57.083+0000 7fa3af9f4700 1 --2- 172.21.15.71:0/2664363309 >> [v2:172.21.15.64:6800/1408391390,v1:172.21.15.64:6801/1408391390] conn(0x7fa394031090 0x7fa394033550 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T13:02:57.084 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:57.083+0000 7fa3af9f4700 1 --2- 172.21.15.71:0/2664363309 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa3a80602c0 0x7fa3a8106730 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-20T13:02:57.084 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:57.083+0000 7fa3af9f4700 1 -- 172.21.15.71:0/2664363309 >> 172.21.15.71:0/2664363309 conn(0x7fa3a80f0be0 msgr2=0x7fa3a80f1630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-20T13:02:57.085 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:57.083+0000 7fa3af9f4700 1 -- 172.21.15.71:0/2664363309 shutdown_connections 2024-02-20T13:02:57.085 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-20T13:02:57.083+0000 7fa3af9f4700 1 -- 172.21.15.71:0/2664363309 wait complete. 2024-02-20T13:02:57.085 INFO:teuthology.orchestra.run.smithi071.stderr:dumped monmap epoch 1 2024-02-20T13:02:57.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 13:02:57 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: cluster 2024-02-20T13:02:56.113360+0000 mgr.smithi064.vriyor (mgr.14156) 378 : cluster [DBG] pgmap v356: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-20T13:02:57.562 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 13:02:57 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: audit 2024-02-20T13:02:57.082582+0000 mon.smithi064 (mon.0) 336 : audit [DBG] from='client.? 172.21.15.71:0/2664363309' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-20T13:02:57.902 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/contextutil.py", line 30, in nested vars.append(enter()) File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/github.com_ceph_ceph-c_b0bd01b8442d66b8a31f8cb3050334b026de2f3b/qa/tasks/cephadm.py", line 675, in ceph_mons while proceed(): File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (181) after waiting for 180 seconds 2024-02-20T13:02:57.903 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-02-20T13:02:57.904 DEBUG:teuthology.orchestra.run.smithi064:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-02-20T13:02:57.927 DEBUG:teuthology.orchestra.run.smithi071:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-02-20T13:02:57.946 INFO:tasks.cephadm:Stopping all daemons... 2024-02-20T13:02:57.946 INFO:tasks.cephadm.mon.smithi064:Stopping mon.smithi064... 2024-02-20T13:02:57.946 DEBUG:teuthology.orchestra.run.smithi064:> sudo systemctl stop ceph-680ee126-cfee-11ee-95bb-87774f69a715@mon.smithi064 2024-02-20T13:02:58.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 13:02:58 smithi064 systemd[1]: Stopping Ceph mon.smithi064 for 680ee126-cfee-11ee-95bb-87774f69a715... 2024-02-20T13:02:58.312 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 13:02:58 smithi064 podman[44812]: Error: no container with name or ID "ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon.smithi064" found: no such container 2024-02-20T13:02:58.812 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 13:02:58 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: debug 2024-02-20T13:02:58.335+0000 7f9d2c3c2700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi064 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-stderr=true --default-log-stderr-prefix=debug --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-stderr=true (PID: 1) UID: 0 2024-02-20T13:02:58.812 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 13:02:58 smithi064 ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064[38353]: debug 2024-02-20T13:02:58.335+0000 7f9d2c3c2700 -1 mon.smithi064@0(leader) e1 *** Got Signal Terminated *** 2024-02-20T13:02:59.491 INFO:journalctl@ceph.mon.smithi064.smithi064.stdout:Feb 20 13:02:59 smithi064 bash[44852]: ceph-680ee126-cfee-11ee-95bb-87774f69a715-mon-smithi064 2024-02-20T13:02:59.497 DEBUG:teuthology.orchestra.run.smithi064:> sudo pkill -f 'journalctl -f -n 0 -u ceph-680ee126-cfee-11ee-95bb-87774f69a715@mon.smithi064.service' 2024-02-20T13:02:59.552 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-20T13:02:59.552 INFO:tasks.cephadm.mon.smithi064:Stopped mon.smithi064 2024-02-20T13:02:59.553 INFO:tasks.cephadm.mon.smithi071:Stopping mon.smithi071... 2024-02-20T13:02:59.553 DEBUG:teuthology.orchestra.run.smithi071:> sudo systemctl stop ceph-680ee126-cfee-11ee-95bb-87774f69a715@mon.smithi071 2024-02-20T13:02:59.591 INFO:teuthology.orchestra.run.smithi071.stderr:Failed to stop ceph-680ee126-cfee-11ee-95bb-87774f69a715@mon.smithi071.service: Unit ceph-680ee126-cfee-11ee-95bb-87774f69a715@mon.smithi071.service not loaded. 2024-02-20T13:02:59.592 DEBUG:teuthology.orchestra.run:got remote process result: 5 2024-02-20T13:02:59.593 ERROR:tasks.cephadm:Failed to stop "ceph.mon.smithi071" Traceback (most recent call last): File "/home/teuthworker/src/github.com_ceph_ceph-c_b0bd01b8442d66b8a31f8cb3050334b026de2f3b/qa/tasks/cephadm.py", line 549, in ceph_bootstrap yield File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/contextutil.py", line 30, in nested vars.append(enter()) File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/github.com_ceph_ceph-c_b0bd01b8442d66b8a31f8cb3050334b026de2f3b/qa/tasks/cephadm.py", line 675, in ceph_mons while proceed(): File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (181) after waiting for 180 seconds During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/github.com_ceph_ceph-c_b0bd01b8442d66b8a31f8cb3050334b026de2f3b/qa/tasks/cephadm.py", line 568, in ceph_bootstrap ctx.daemons.get_daemon(type_, id_, cluster).stop() File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/orchestra/daemon/cephadmunit.py", line 154, in stop self.remote.sh(self.stop_cmd) File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/orchestra/remote.py", line 97, in sh proc = self.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi071 with status 5: 'sudo systemctl stop ceph-680ee126-cfee-11ee-95bb-87774f69a715@mon.smithi071' 2024-02-20T13:02:59.594 INFO:tasks.cephadm:Archiving crash dumps... 2024-02-20T13:02:59.596 DEBUG:teuthology.misc:Transferring archived files from smithi064:/var/lib/ceph/680ee126-cfee-11ee-95bb-87774f69a715/crash to /home/teuthworker/archive/yuriw-2024-02-19_20:21:18-fs-wip-yuri5-testing-2024-02-16-1120-quincy-distro-default-smithi/7567673/remote/smithi064/crash 2024-02-20T13:02:59.597 DEBUG:teuthology.orchestra.run.smithi064:> sudo tar c -f - -C /var/lib/ceph/680ee126-cfee-11ee-95bb-87774f69a715/crash -- . 2024-02-20T13:02:59.646 DEBUG:teuthology.misc:Transferring archived files from smithi071:/var/lib/ceph/680ee126-cfee-11ee-95bb-87774f69a715/crash to /home/teuthworker/archive/yuriw-2024-02-19_20:21:18-fs-wip-yuri5-testing-2024-02-16-1120-quincy-distro-default-smithi/7567673/remote/smithi071/crash 2024-02-20T13:02:59.647 DEBUG:teuthology.orchestra.run.smithi071:> sudo tar c -f - -C /var/lib/ceph/680ee126-cfee-11ee-95bb-87774f69a715/crash -- . 2024-02-20T13:02:59.703 INFO:teuthology.orchestra.run.smithi071.stderr:tar: /var/lib/ceph/680ee126-cfee-11ee-95bb-87774f69a715/crash: Cannot open: No such file or directory 2024-02-20T13:02:59.703 INFO:teuthology.orchestra.run.smithi071.stderr:tar: Error is not recoverable: exiting now 2024-02-20T13:02:59.707 INFO:tasks.cephadm:Checking cluster log for badness... 2024-02-20T13:02:59.707 DEBUG:teuthology.orchestra.run.smithi064:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/680ee126-cfee-11ee-95bb-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | head -n 1 2024-02-20T13:02:59.740 INFO:teuthology.orchestra.run.smithi064.stderr:grep: /var/log/ceph/680ee126-cfee-11ee-95bb-87774f69a715/ceph.log: No such file or directory 2024-02-20T13:02:59.742 INFO:tasks.cephadm:Compressing logs... 2024-02-20T13:02:59.742 DEBUG:teuthology.orchestra.run.smithi064:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-02-20T13:02:59.786 DEBUG:teuthology.orchestra.run.smithi071:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-02-20T13:02:59.813 INFO:teuthology.orchestra.run.smithi064.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-02-20T13:02:59.813 INFO:teuthology.orchestra.run.smithi064.stderr:: No such file or directory 2024-02-20T13:02:59.814 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/680ee126-cfee-11ee-95bb-87774f69a715/ceph-volume.log 2024-02-20T13:02:59.815 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/680ee126-cfee-11ee-95bb-87774f69a715/ceph-mgr.smithi064.vriyor.log 2024-02-20T13:02:59.815 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/680ee126-cfee-11ee-95bb-87774f69a715/ceph-mon.smithi064.log 2024-02-20T13:02:59.815 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/680ee126-cfee-11ee-95bb-87774f69a715/ceph-volume.log: /var/log/ceph/680ee126-cfee-11ee-95bb-87774f69a715/ceph-mgr.smithi064.vriyor.log: /var/log/ceph/680ee126-cfee-11ee-95bb-87774f69a715/ceph-mon.smithi064.log: 90.2% -- replaced with /var/log/ceph/680ee126-cfee-11ee-95bb-87774f69a715/ceph-volume.log.gz 2024-02-20T13:02:59.816 INFO:teuthology.orchestra.run.smithi071.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-02-20T13:02:59.816 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-02-20T13:02:59.816 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/680ee126-cfee-11ee-95bb-87774f69a715/ceph-volume.log 2024-02-20T13:02:59.817 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/cephadm.log: /var/log/ceph/680ee126-cfee-11ee-95bb-87774f69a715/ceph-volume.log: 88.4% -- replaced with /var/log/ceph/680ee126-cfee-11ee-95bb-87774f69a715/ceph-volume.log.gz 2024-02-20T13:02:59.820 INFO:teuthology.orchestra.run.smithi064.stderr: 92.9% -- replaced with /var/log/ceph/cephadm.log.gz 2024-02-20T13:02:59.821 INFO:teuthology.orchestra.run.smithi071.stderr: 95.2% -- replaced with /var/log/ceph/cephadm.log.gz 2024-02-20T13:02:59.822 INFO:teuthology.orchestra.run.smithi071.stderr: 2024-02-20T13:02:59.822 INFO:teuthology.orchestra.run.smithi071.stderr:real 0m0.018s 2024-02-20T13:02:59.822 INFO:teuthology.orchestra.run.smithi071.stderr:user 0m0.016s 2024-02-20T13:02:59.822 INFO:teuthology.orchestra.run.smithi071.stderr:sys 0m0.012s 2024-02-20T13:02:59.911 INFO:teuthology.orchestra.run.smithi064.stderr:gzip: /var/log/ceph/680ee126-cfee-11ee-95bb-87774f69a715/ceph-mgr.smithi064.vriyor.log: file size changed while zipping 2024-02-20T13:02:59.912 INFO:teuthology.orchestra.run.smithi064.stderr: 92.8% -- replaced with /var/log/ceph/680ee126-cfee-11ee-95bb-87774f69a715/ceph-mgr.smithi064.vriyor.log.gz 2024-02-20T13:02:59.992 INFO:teuthology.orchestra.run.smithi064.stderr: 90.6% -- replaced with /var/log/ceph/680ee126-cfee-11ee-95bb-87774f69a715/ceph-mon.smithi064.log.gz 2024-02-20T13:02:59.993 INFO:teuthology.orchestra.run.smithi064.stderr: 2024-02-20T13:02:59.993 INFO:teuthology.orchestra.run.smithi064.stderr:real 0m0.191s 2024-02-20T13:02:59.993 INFO:teuthology.orchestra.run.smithi064.stderr:user 0m0.279s 2024-02-20T13:02:59.993 INFO:teuthology.orchestra.run.smithi064.stderr:sys 0m0.027s 2024-02-20T13:02:59.994 INFO:tasks.cephadm:Archiving logs... 2024-02-20T13:02:59.994 DEBUG:teuthology.misc:Transferring archived files from smithi064:/var/log/ceph to /home/teuthworker/archive/yuriw-2024-02-19_20:21:18-fs-wip-yuri5-testing-2024-02-16-1120-quincy-distro-default-smithi/7567673/remote/smithi064/log 2024-02-20T13:02:59.995 DEBUG:teuthology.orchestra.run.smithi064:> sudo tar c -f - -C /var/log/ceph -- . 2024-02-20T13:03:00.118 DEBUG:teuthology.misc:Transferring archived files from smithi071:/var/log/ceph to /home/teuthworker/archive/yuriw-2024-02-19_20:21:18-fs-wip-yuri5-testing-2024-02-16-1120-quincy-distro-default-smithi/7567673/remote/smithi071/log 2024-02-20T13:03:00.119 DEBUG:teuthology.orchestra.run.smithi071:> sudo tar c -f - -C /var/log/ceph -- . 2024-02-20T13:03:00.154 INFO:tasks.cephadm:Removing cluster... 2024-02-20T13:03:00.154 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 680ee126-cfee-11ee-95bb-87774f69a715 --force 2024-02-20T13:07:12.354 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 680ee126-cfee-11ee-95bb-87774f69a715 --force 2024-02-20T13:07:12.809 INFO:tasks.cephadm:Removing cephadm ... 2024-02-20T13:07:12.810 DEBUG:teuthology.orchestra.run.smithi064:> rm -rf /home/ubuntu/cephtest/cephadm 2024-02-20T13:07:12.830 DEBUG:teuthology.orchestra.run.smithi071:> rm -rf /home/ubuntu/cephtest/cephadm 2024-02-20T13:07:12.849 INFO:tasks.cephadm:Teardown complete 2024-02-20T13:07:12.849 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/github.com_ceph_ceph-c_b0bd01b8442d66b8a31f8cb3050334b026de2f3b/qa/tasks/cephadm.py", line 549, in ceph_bootstrap yield File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/contextutil.py", line 30, in nested vars.append(enter()) File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/github.com_ceph_ceph-c_b0bd01b8442d66b8a31f8cb3050334b026de2f3b/qa/tasks/cephadm.py", line 675, in ceph_mons while proceed(): File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (181) after waiting for 180 seconds During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/github.com_ceph_ceph-c_b0bd01b8442d66b8a31f8cb3050334b026de2f3b/qa/tasks/cephadm.py", line 1564, in task with contextutil.nested( File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/contextutil.py", line 54, in nested raise exc[1] File "/usr/lib/python3.8/contextlib.py", line 131, in __exit__ self.gen.throw(type, value, traceback) File "/home/teuthworker/src/github.com_ceph_ceph-c_b0bd01b8442d66b8a31f8cb3050334b026de2f3b/qa/tasks/cephadm.py", line 1477, in initialize_config yield File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/contextutil.py", line 46, in nested if exit(*exc): File "/usr/lib/python3.8/contextlib.py", line 131, in __exit__ self.gen.throw(type, value, traceback) File "/home/teuthworker/src/github.com_ceph_ceph-c_b0bd01b8442d66b8a31f8cb3050334b026de2f3b/qa/tasks/cephadm.py", line 568, in ceph_bootstrap ctx.daemons.get_daemon(type_, id_, cluster).stop() File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/orchestra/daemon/cephadmunit.py", line 154, in stop self.remote.sh(self.stop_cmd) File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/orchestra/remote.py", line 97, in sh proc = self.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi071 with status 5: 'sudo systemctl stop ceph-680ee126-cfee-11ee-95bb-87774f69a715@mon.smithi071' 2024-02-20T13:07:13.084 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=6dcef11f8207464bbf2bd9f474d721aa Traceback (most recent call last): File "/home/teuthworker/src/github.com_ceph_ceph-c_b0bd01b8442d66b8a31f8cb3050334b026de2f3b/qa/tasks/cephadm.py", line 549, in ceph_bootstrap yield File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/contextutil.py", line 30, in nested vars.append(enter()) File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/github.com_ceph_ceph-c_b0bd01b8442d66b8a31f8cb3050334b026de2f3b/qa/tasks/cephadm.py", line 675, in ceph_mons while proceed(): File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (181) after waiting for 180 seconds During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/github.com_ceph_ceph-c_b0bd01b8442d66b8a31f8cb3050334b026de2f3b/qa/tasks/cephadm.py", line 1564, in task with contextutil.nested( File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/contextutil.py", line 54, in nested raise exc[1] File "/usr/lib/python3.8/contextlib.py", line 131, in __exit__ self.gen.throw(type, value, traceback) File "/home/teuthworker/src/github.com_ceph_ceph-c_b0bd01b8442d66b8a31f8cb3050334b026de2f3b/qa/tasks/cephadm.py", line 1477, in initialize_config yield File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/contextutil.py", line 46, in nested if exit(*exc): File "/usr/lib/python3.8/contextlib.py", line 131, in __exit__ self.gen.throw(type, value, traceback) File "/home/teuthworker/src/github.com_ceph_ceph-c_b0bd01b8442d66b8a31f8cb3050334b026de2f3b/qa/tasks/cephadm.py", line 568, in ceph_bootstrap ctx.daemons.get_daemon(type_, id_, cluster).stop() File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/orchestra/daemon/cephadmunit.py", line 154, in stop self.remote.sh(self.stop_cmd) File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/orchestra/remote.py", line 97, in sh proc = self.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi071 with status 5: 'sudo systemctl stop ceph-680ee126-cfee-11ee-95bb-87774f69a715@mon.smithi071' 2024-02-20T13:07:13.089 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-02-20T13:07:13.099 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-02-20T13:07:13.113 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-02-20T13:07:13.114 DEBUG:teuthology.orchestra.run.smithi064:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-02-20T13:07:13.132 DEBUG:teuthology.orchestra.run.smithi071:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-02-20T13:07:13.216 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-02-20T13:07:13.225 INFO:teuthology.task.clock:Checking final clock skew... 2024-02-20T13:07:13.225 DEBUG:teuthology.orchestra.run.smithi064:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-02-20T13:07:13.228 DEBUG:teuthology.orchestra.run.smithi071:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-02-20T13:07:13.245 INFO:teuthology.orchestra.run.smithi071.stderr:bash: ntpq: command not found 2024-02-20T13:07:13.245 INFO:teuthology.orchestra.run.smithi064.stderr:bash: ntpq: command not found 2024-02-20T13:07:13.249 INFO:teuthology.orchestra.run.smithi071.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-02-20T13:07:13.249 INFO:teuthology.orchestra.run.smithi071.stdout:=============================================================================== 2024-02-20T13:07:13.249 INFO:teuthology.orchestra.run.smithi071.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 17 +1093us[+1093us] +/- 132ms 2024-02-20T13:07:13.249 INFO:teuthology.orchestra.run.smithi071.stdout:^* hv02.front.sepia.ceph.com 3 6 377 20 -397us[ -376us] +/- 54ms 2024-02-20T13:07:13.249 INFO:teuthology.orchestra.run.smithi071.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 20 +66us[ +66us] +/- 92ms 2024-02-20T13:07:13.249 INFO:teuthology.orchestra.run.smithi071.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-02-20T13:07:13.251 INFO:teuthology.orchestra.run.smithi064.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-02-20T13:07:13.251 INFO:teuthology.orchestra.run.smithi064.stdout:=============================================================================== 2024-02-20T13:07:13.251 INFO:teuthology.orchestra.run.smithi064.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 17 +1050us[+1050us] +/- 132ms 2024-02-20T13:07:13.251 INFO:teuthology.orchestra.run.smithi064.stdout:^* hv02.front.sepia.ceph.com 3 6 377 18 -410us[ -399us] +/- 54ms 2024-02-20T13:07:13.251 INFO:teuthology.orchestra.run.smithi064.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 18 +36us[ +47us] +/- 92ms 2024-02-20T13:07:13.251 INFO:teuthology.orchestra.run.smithi064.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-02-20T13:07:13.252 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-02-20T13:07:13.261 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-02-20T13:07:13.262 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-02-20T13:07:13.271 DEBUG:teuthology.orchestra.run.smithi064:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-02-20T13:07:13.359 DEBUG:teuthology.orchestra.run.smithi071:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-02-20T13:07:13.424 DEBUG:teuthology.orchestra.run.smithi064:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-02-20T13:07:13.456 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-20T13:07:13.457 DEBUG:teuthology.orchestra.run.smithi071:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-02-20T13:07:13.486 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-20T13:07:13.487 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-02-20T13:07:13.497 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-02-20T13:07:13.506 INFO:teuthology.task.internal:Duration was 1676.937275 seconds 2024-02-20T13:07:13.506 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-02-20T13:07:13.516 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-02-20T13:07:13.517 DEBUG:teuthology.orchestra.run.smithi064:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-02-20T13:07:13.519 DEBUG:teuthology.orchestra.run.smithi071:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-02-20T13:07:13.567 INFO:teuthology.orchestra.run.smithi064.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-02-20T13:07:13.573 INFO:teuthology.orchestra.run.smithi071.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-02-20T13:07:13.972 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-02-20T13:07:13.972 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi064.front.sepia.ceph.com 2024-02-20T13:07:13.972 DEBUG:teuthology.orchestra.run.smithi064:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-02-20T13:07:14.002 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi071.front.sepia.ceph.com 2024-02-20T13:07:14.002 DEBUG:teuthology.orchestra.run.smithi071:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-02-20T13:07:14.025 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-02-20T13:07:14.026 DEBUG:teuthology.orchestra.run.smithi064:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-02-20T13:07:14.045 DEBUG:teuthology.orchestra.run.smithi071:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-02-20T13:07:14.116 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-02-20T13:07:14.117 DEBUG:teuthology.orchestra.run.smithi064:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-02-20T13:07:14.119 DEBUG:teuthology.orchestra.run.smithi071:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-02-20T13:07:14.210 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-02-20T13:07:14.221 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-02-20T13:07:14.221 DEBUG:teuthology.orchestra.run.smithi064:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-02-20T13:07:14.250 DEBUG:teuthology.orchestra.run.smithi071:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-02-20T13:07:14.277 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-02-20T13:07:14.288 DEBUG:teuthology.orchestra.run.smithi064:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-02-20T13:07:14.293 DEBUG:teuthology.orchestra.run.smithi071:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-02-20T13:07:14.318 INFO:teuthology.orchestra.run.smithi064.stdout:kernel.core_pattern = core 2024-02-20T13:07:14.343 INFO:teuthology.orchestra.run.smithi071.stdout:kernel.core_pattern = core 2024-02-20T13:07:14.368 DEBUG:teuthology.orchestra.run.smithi064:> test -e /home/ubuntu/cephtest/archive/coredump 2024-02-20T13:07:14.406 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-20T13:07:14.407 DEBUG:teuthology.orchestra.run.smithi071:> test -e /home/ubuntu/cephtest/archive/coredump 2024-02-20T13:07:14.425 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-20T13:07:14.425 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-02-20T13:07:14.436 INFO:teuthology.task.internal:Transferring archived files... 2024-02-20T13:07:14.437 DEBUG:teuthology.misc:Transferring archived files from smithi064:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/yuriw-2024-02-19_20:21:18-fs-wip-yuri5-testing-2024-02-16-1120-quincy-distro-default-smithi/7567673/remote/smithi064 2024-02-20T13:07:14.437 DEBUG:teuthology.orchestra.run.smithi064:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-02-20T13:07:14.489 DEBUG:teuthology.misc:Transferring archived files from smithi071:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/yuriw-2024-02-19_20:21:18-fs-wip-yuri5-testing-2024-02-16-1120-quincy-distro-default-smithi/7567673/remote/smithi071 2024-02-20T13:07:14.489 DEBUG:teuthology.orchestra.run.smithi071:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-02-20T13:07:14.529 INFO:teuthology.task.internal:Removing archive directory... 2024-02-20T13:07:14.529 DEBUG:teuthology.orchestra.run.smithi064:> rm -rf -- /home/ubuntu/cephtest/archive 2024-02-20T13:07:14.531 DEBUG:teuthology.orchestra.run.smithi071:> rm -rf -- /home/ubuntu/cephtest/archive 2024-02-20T13:07:14.580 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-02-20T13:07:14.591 INFO:teuthology.task.internal:Not uploading archives. 2024-02-20T13:07:14.591 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-02-20T13:07:14.602 INFO:teuthology.task.internal:Tidying up after the test... 2024-02-20T13:07:14.602 DEBUG:teuthology.orchestra.run.smithi064:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-02-20T13:07:14.605 DEBUG:teuthology.orchestra.run.smithi071:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-02-20T13:07:14.620 INFO:teuthology.orchestra.run.smithi064.stdout: 526077 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Feb 20 13:07 /home/ubuntu/cephtest 2024-02-20T13:07:14.638 INFO:teuthology.orchestra.run.smithi071.stdout: 525357 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Feb 20 13:07 /home/ubuntu/cephtest 2024-02-20T13:07:14.640 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-02-20T13:07:14.651 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-02-20T13:07:14.672 INFO:teuthology.nuke:Checking targets against current locks 2024-02-20T13:07:14.700 DEBUG:teuthology.nuke:shortname: smithi064 2024-02-20T13:07:14.701 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-02-20T13:07:14.724 DEBUG:teuthology.nuke:shortname: smithi071 2024-02-20T13:07:14.724 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-02-20T13:07:14.728 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi064.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2024-02-19_20:21:18-fs-wip-yuri5-testing-2024-02-16-1120-quincy-distro-default-smithi/7567673', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '8', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-02-20 12:30:56.879902', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBaltUs4d994y8I+1xeyo/rx86HpvBqhma4wgetKpfmdAPu4CPm4wVVKw08FRyoFJ6PLXHBKHNnadG75dXQEtqI='} 2024-02-20T13:07:14.749 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi071.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2024-02-19_20:21:18-fs-wip-yuri5-testing-2024-02-16-1120-quincy-distro-default-smithi/7567673', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '8', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-02-20 12:30:56.881907', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBC5alVGgGW1Ddqv3KW7HbscCa8SQnau0YCExXl3AACgrxv29h791W7mrH+v+6U9V83I/9+z8IkK189a2j34PrHc='} 2024-02-20T13:07:14.782 INFO:teuthology.orchestra.console.smithi064:Power off 2024-02-20T13:07:14.783 DEBUG:teuthology.orchestra.console.smithi064:pexpect command: ipmitool -H smithi064.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-02-20T13:07:14.808 DEBUG:teuthology.orchestra.console.smithi064:power off output: Chassis Power Control: Down/Off 2024-02-20T13:07:14.808 DEBUG:teuthology.orchestra.console.smithi064:pexpect command: ipmitool -H smithi064.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-20T13:07:14.825 INFO:teuthology.orchestra.console.smithi071:Power off 2024-02-20T13:07:14.826 DEBUG:teuthology.orchestra.console.smithi071:pexpect command: ipmitool -H smithi071.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-02-20T13:07:14.840 DEBUG:teuthology.orchestra.console.smithi064:check power output: Chassis Power is on 2024-02-20T13:07:14.849 DEBUG:teuthology.orchestra.console.smithi071:power off output: Chassis Power Control: Down/Off 2024-02-20T13:07:14.849 DEBUG:teuthology.orchestra.console.smithi071:pexpect command: ipmitool -H smithi071.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-20T13:07:14.873 DEBUG:teuthology.orchestra.console.smithi071:check power output: Chassis Power is on 2024-02-20T13:07:18.841 DEBUG:teuthology.orchestra.console.smithi064:pexpect command: ipmitool -H smithi064.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-20T13:07:18.875 DEBUG:teuthology.orchestra.console.smithi071:pexpect command: ipmitool -H smithi071.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-20T13:07:18.952 DEBUG:teuthology.orchestra.console.smithi064:check power output: Chassis Power is on 2024-02-20T13:07:18.985 DEBUG:teuthology.orchestra.console.smithi071:check power output: Chassis Power is on 2024-02-20T13:07:22.956 DEBUG:teuthology.orchestra.console.smithi064:pexpect command: ipmitool -H smithi064.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-20T13:07:22.985 DEBUG:teuthology.orchestra.console.smithi071:pexpect command: ipmitool -H smithi071.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-20T13:07:23.069 DEBUG:teuthology.orchestra.console.smithi064:check power output: Chassis Power is off 2024-02-20T13:07:23.095 DEBUG:teuthology.orchestra.console.smithi071:check power output: Chassis Power is on 2024-02-20T13:07:23.170 INFO:teuthology.orchestra.console.smithi064:Power off completed 2024-02-20T13:07:27.100 DEBUG:teuthology.orchestra.console.smithi071:pexpect command: ipmitool -H smithi071.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-20T13:07:27.210 DEBUG:teuthology.orchestra.console.smithi071:check power output: Chassis Power is off 2024-02-20T13:07:27.311 INFO:teuthology.orchestra.console.smithi071:Power off completed 2024-02-20T13:07:27.418 INFO:teuthology.run:Summary data: description: fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/pacific 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-inline/no 4-verify} 2-client 3-upgrade-with-workload 4-verify}} duration: 1676.9372746944427 failure_reason: 'Command failed on smithi071 with status 5: ''sudo systemctl stop ceph-680ee126-cfee-11ee-95bb-87774f69a715@mon.smithi071''' flavor: default owner: scheduled_yuriw@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=6dcef11f8207464bbf2bd9f474d721aa status: fail success: false 2024-02-20T13:07:27.419 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-02-20T13:07:27.510 INFO:teuthology.run:FAIL